Are botnets run by spy agencies?

A recent story today about discussions for an official defense Botnet in the USA prompted me to post a question I’ve been asking for the last year. Are some of the world’s botnets secretly run by intelligence agencies, and if not, why not?

Some estimates suggest that up to 1/3 of PCs are secretly part of a botnet. The main use of botnets is sending spam, but they are also used for DDOS extortion attacks and presumably other nasty things like identity theft.

But consider this — having remote control of millions of PCs, and a large percentage of the world’s PCs seems like a very tempting target for the world’s various intelligence agencies. Most zombies are used for external purposes, but it would be easy to have them searching their own disk drives for interesting documents, and sniffing their own LANs for interesting unencrypted LAN traffic, or using their internal state to get past firewalls.

Considering the billions that spy agencies like the NSA, MI6, CSEC and others spend on getting a chance to sniff signals as they go over the wires, being able to look at the data all the time, any time as it sits on machines must be incredibly tempting.

And if the botnet lore is to be accepted, all this was done using the resources of a small group of young intrusion experts. If a group of near kids can control hundreds of millions of machines, should not security experts with billions of dollars be tempted to do it?

Of course there are legal/treaty issues. Most “free nation” spy agencies are prohibited from breaking into computers in their own countries without a warrant. (However, as we’ve seen, the NSA has recently been lifted of this restriction, and we’re suing over that.) However, they are not restricted on what they do to foreign computers, other than by the burdens of keeping up good relations with our allies.

However, in some cases the ECHELON loophole may be used, where the NSA spies on British computers and MI-6 spies on American computers in exchange.

More simply, these spy agencies would not want to get caught at this, so they would want to use young hackers building spam-networks as a front. They would be very careful to assure that the botting could not be traced back to them. To keep it legal, they might even just not take information from computers whose IP addresses or other clues suggest they are domestic. The criminal botnet operators could infect everywhere, but the spies would be more careful about where they got information and what they paid for.

Of course, spy agencies of many countries would suffer no such restrictions on domestic spying.

Of all the spy agencies in the world, can it be that none of them have thought of this? That none of them are tempted by being able to comb through a large fraction of the world’s disk drives, looking for both bad guys and doing plain old espionage?

That’s hard to fathom. The question is, how would we detect it? And if it’s true, could it mean that spies funded (as a cover story) the world’s spamming infrastructure?

Spam turns 30, The eCheck is in the eMail

Been getting a bunch of calls from reporters this weekend. Our good friend spam turns 30 in a couple of days, and a few years ago I did some research and became an authority on the history of the term and the phenomenon. Since everybody else is doing it, I though I should point to my various articles on the history of spam, as well as some updates I just wrote for the 30th.

If you’ve seen all this before, you can mostly focus on the new thoughts where I talk about the rise of Botnets — which may negate many of the best anti-spam solutions, and the flight to Facebook from e-mail by the younger generation.

Update: I’ve done a bunch of press interviews on spam this week, and was on May 3’s “Weekend All Things Considered” on NPR as well. I get in quite a few words, especially for radio.

Spam Filters and the degradation of e-Mail.

And now this new thought. Spam filters are working better, but content filtering means false positives. Curiously, this has brought an unreliability to e-mail which parallels the famous (but mostly false) unreliability of the postal service.

In the old days, the widespread belief in the poor quality of the postal service was a popular excuse. You could always tell somebody you hadn’t received his letter, or that it had been very slow in arriving, or that you had sent your letter but it must have gotten lost or delayed. And people would either believe it, or feel they had to pretend to believe it. We dubbed the system “snail mail” to reiterate this, and of course “The Cheque is in the Mail” became known as one of the 3 great lies.

For a while e-mail was good enough that you couldn’t use this lie so easily. And it was too fast, as well. The only question was when you would get around to reading your mail.

But now we’ve got a new excuse — your mail got trashed by my spam filters. Oh, wait, I found it, in my spam folder. A nice and convenient lie that the other party can’t quite call you out on. Likewise, now, when I send a mail and don’t hear back, I always have to wonder if the mail has been caught in the filters.

(More often than not, non-response is due to another e-mail phenomenon: the growing stack. If I can’t answer your mail right away there is a danger it will move down the stack in my e-mail box, and soon be lost to attention, even though it’s there and I read it.)

29th anniversary of spam

I wasn’t going to make any special commemoration, but it seems a whole ton of other blogs are linking today to my articles on the history of Spam, so I should blog them as well.

Many years ago I got interested in the origins of the term “spam” to mean net abuse. I mean I had lived through most of its origin and seen most of the early spams myself, but it wasn’t clear why people took the name of the meat product and applied it to junk mail. I knew it came from USENET, so I used the USENET search engines to trace the origins.

This resulted in my article on the origins of word spam to mean net abuse.

In doing the research, I was pointed to what was probably the earliest internet spam, though it far predates the term.

I documented that in Reactions to the first spam.

4 years ago, on the 25th anniversary of that spam, I was interviewed on NPR’s All Things Considered and write an article reflecting on the history. For that article I dug out Gary Thuerk, the sender of that first spam, and interviewed him for more details.

You can read that in Reflections on the 25th anniversary of Spam.

Of course, you can find all these and many more in my collection of articles on Spam. Many years ago I wrote a wide variety of essays on the spam problem. Not simply about solutions, but analysis of why the fight was so nasty, and concern over the rights people were willing to give up in the name of fighting spam.

I will probably update them, and do some more research for the 30th anniversary, next year.

Tempfailing for spam -- where does it lead

One growing technique for use in anti-spam involves finding ways to “fail” on initial contacts for sending mail. Real, standard conformant mail programs try again in various ways, but spammers, in writing their mail blasters, tend to just have them skip that address and go to the next one in their list.

Two common approaches include simply returning a “temporarily unavailable” status on any initial mail attempt that might be spam. Another approach is to have dead MX records both at the “try first” and “try last” end of the MX chain.

Why does this work? Spammers just want to deliver as much mail as possible given time and bandwidth. If one address fails for any reason, it’s really no different whether you spend your resources trying the address again or in a different way, or just move on to the next address. In fact, since many of the failures are real failures, it’s actually more productive to just move on.

And, I admit, some of the spam filtering tools I make use of use these techniques, and they do help. But what exactly are they doing? For spammers, the limiting factor is bandwidth. Dealing with failures, especially timeouts on dead servers, takes very little of their resources.

It doesn’t reduce the amount of spam they send, at least by much, it just redistributes it to those who don’t use the techniques. For a positive spin, you can liken it to putting up a higher fence than your neighbour, so the criminals attack them and not you. For a negative spin, you can imagine it as being like an air filter that filters out the pollution on air coming into your house, and spews it out the back at your neighbours.

So it’s a touch question. Is this approach a good idea? Especially at the start, it was very effective. Over time if it becomes very common spammers will see a reduction in spam they deliver and make fairly simple moves to compensate for it. Is this fair game or antisocial?

There is an old joke about two hikers who meet a bear. The first sits down and starts putting on his running shoes. The other says, “What are you doing, you can’t outrun a bear!” and the first says, “I don’t have to outrun the bear, I just have to outrun you.”

Are we passing the bear onto our neighbours?

(This is part of a larger question of some of the other negative consequences of anti-spam. For example, as text filters got better, spammers moved to sending their spam as embedded images which filters could not easily decode. The result is more and more bandwidth used, both by spammers and victims. Was it a victory or a loss?)

Now you have to have the right reverse-DNS

Update: Several of the spam bounces of this sort that I got were traced to the same anti-spam system, and the operator says it was not intentional, and has been corrected. So it may not be quite as bad as it seemed quite yet.

I have a social list of people I invite to parties. Every time I mail to it, I feel the impact of spam and anti-spam. Always several people have given up on a mailbox. And I run into new spam filters blocking the mail.

Perhaps I’m an old timer, but I run my own mail server. It’s in my house. I read my mail on that actual machine, and because of that, mail is wicked-fast for me, as fast as instant messaging for many people. (In fact, I never adopted IM the way some people did because E-mail is as fast.)

They’re working to make this harder to do. Many ISPs won’t even let you send mail directly, or demand you make a special request to have the mail port open to you. I’m bothered by the first case, less so by the second, because indeed, zombie PCs send much of the spam we’re now getting.

Because I send mail from the system, I also web surf from it. And while it’s not a serious privacy protection, I decided I would not have a reverse-DNS record for my system. That way people would not see “” in their web logs whenever I surfed. It’s not that you can’t use other techniques to find out that the address is mine, but that requires deliberate thought. Reverse DNS is automatic for many web logs.

Soon more and more sites would not take mail from a system without reverse DNS. Because I get my IP block from a small ISP, he does my reverse DNS, and I asked him to make one. He made one like many ISPs do, built from the IP numbers themselves. As in

But soon I saw bounces that said, “This reverse DNS looks like a dialup user, I won’t take your mail.” So I had him change it to a different string that doesn’t trumpet my name but doesn’t look like a standard anonymous reverse DNS.

But now I’m getting bounces just because the reverse DNS doesn’t match the name my mail server uses. There is no security in this, any spammer can program their mail server to use the reverse DNS name of the system they have taken over. But I guess some don’t, so another wall is thrown up, and those people won’t get invites to my parties.

This one is really stupid because it’s quite common for a single machine to have many names and serve many domains. To correct an earlier note, it is possible for an IP to have more than one PTR reverse DNS record, though I don’t know how many applications deal with that. And that screws these mailers. There is no need to look at reverse DNS at all.


The comment spammers are going manual, it seems

Some time ago I modified this blog softare (Drupal) to ask a very simple question of people without accounts posting comments. It generally works very well at stopping robot posting, however the volume of spam has been increasing, so I changed the question. Volume may have dropped a touch but I still got a bunch, which means the spammers are actually live humans, not robots.

It’s also possible that asking natural language questions (rather than captcha style entry of text from a graphic) has gotten common enough that spammers have modified their software so they can figure out the answer once and easily code it, but I don’t think this is the case.

What’s curious is that my comment form also clearly explains that any links in comments will be done with the rel=nofollow tag, which tells Google and other search engines not to treat the link as a valid one when ranking pages. This means that, other than readers of the blog clicking on the links, which should be very rare, these spams should be unproductive for the spammer. But they’re still doing them.

The change however was prompted by a new breed of comment spam, where the spammers were copying other comments from inside large threads, but inserting their link on the author’s name. (This also uses rel=nofollow.) Indeed, such a technique does not automatically trigger my instincts to delete the spam, but they chose one of my own comments, so I recognized it. Right now my methods cut the spam enough that it is productive to manually delete what gets posted, though if the volume got high enough I would have to find other automated techniques.

(Drupal could of course help by having a much easier to use delete, including a ‘delete all from this IP address’ option.)

EFF Debate on Charging for E-mail Dyson v. O'Brien in SF

TONIGHT, April 20th, there will be a debate on the issue of per-message charges for E-mail, sparked by the recent debate over Goodmail and AOL.

The debate will feature former EFF Chair Esther Dyson, who has become a surprising supporter of pay-to-send E-mail, and EFF Activist Danny O’Brien, NTK author and coordinator of EFF’s involvement in the efforts against Goodmail. Esther is also publisher of Release 1.0, host of the PC Forum conference and former chair of ICANN.

Alas, I won’t be able to be there, as I am at a conference out of town, but those who followed the debate in my blog may wish to attend.

EFF will be fundraising, suggested donation $20 but donations are not mandatory.

You can get full details at the BayFF page

Baby Bells announce new "GoodPackets" program to charge for access

New York, March 22, 2006 (CW) Bell South and AT&T, two of the remaining Baby Bell or “iLec” companies announced today, in conjunction with GoodPackets Inc., a program to charge senders for certified delivery of internet packets to their ISP customers.

William Smith, CTO of Bell South, together with AT&T CEO Ed Whitacre, who will be his new boss once the proposed merger is completed, made a joint announcement of the program together with Dick Greengrass, CEO of GoodPackets.

Under the program, customers of GoodPackets interested in better delivery of their packets to AT&T and BellSouth DSL customers will pay GoodPackets a fee to get their packets certified. Certified packets will bypass blocks and filters in the routers of the ISPs for premium delivery to customers, and be tagged as certified to the end-user.

“We’re just seeing too many bad packets these days, and we have to block some of them. But serious, professional sites on the internet don’t want their packets blocked, and are willing to pay to assure they aren’t,” said Whitacre. According to Greengrass, a portion of the money paid to GoodPackets will be given to the ISP in question.”

According to Smith, “his firm should be able, for example, to charge Yahoo Inc. for the opportunity to have its search site load faster than that of Google Inc.”

“A lot of these extra packets filling our pipes are of dubious origin, in any event. A large portion of internet traffic comes from peer to peer filesharing systems which are often infringing copyright, or from companies like Skype bypassing the telcom tarrifs we all have to pay. Charging money will let the legitimate companies out there distinguish their traffic from all this unknown traffic, and assure delivery,” said Whitacre.

Traffic originating from BellSouth and AT&T servers would not need to pay for the premium access. “It’s our network, after all, and our video servers don’t go through the routers to the outside world to get to our users,” said Smith.

Greengrass insisted the fees were not for delivery, but for certification that the packets come from a known and trusted source. Users and ISPs can then decide if they want to give them more reliable delivery and acceptance. That the charges are per packet is simply a way to differentiate the market, and not overcharge low-volume senders.

For those who don’t get it, this is a satire comparing the AOL/Yahoo/Goodmail program to the network neutrality debate.

Demand junk mail by PDF

Who could possibly imagine wanting spam? Well, I just read that in the USA, 100 million trees are felled every year for junk mail. 28 billion gallons of water used to process the paper. And 350 million dollars spent to throw it out. That doesn't include I presume the other costs, including postage and wasted time, this is just the paper part of it.

So I started musing. What if the USPS started making some new rules for bulk mail rates. In particular, that if you want to do bulk mail, you must either use a bonded mailing house, or a special service provided by the post office to which you provide your mailing list. And you MUST provide a PDF or other electronic form of your mailing, with formats for the stupid customizations that they do to mailings. This would simply be the new rule for the bulk pieces.

And then, any household or other address could say, "Give me my bulk mailings in electronic form."
Or possibly fine grain it by sender ID, so that if you want a certain set of senders to be on paper you can specify that, and all others come electronic.

Of course they don't come to your regular mailbox unless you ask. They go to a special mailbox of your choice, perhaps an extra you have or one run by the USPS. Perhaps you go to the USPS web site to see your junk mailings.

All sounds great but of course there are some hairy problems. Obviously shippers would not want to pay the full bulk postage for this, nor should they. However, it is not simply because of the fact that no paper is mailed, it's because people will probably not look at these items as much as they look at their paper junk mail. Like it or not, they spend 50 cents to a dollar for a typical paper junk mailing because they make a profit. However, do they make a profit from the people who would say "don't do it."

In Canada, houses can declare "no flyers" on their mailbox. This stops delivery of bulk flyers, but not mail with postage. It's a start.

The reason the bonded mail houses are needed is that the mailers must not get to learn who is getting PDF and who is getting paper, just how many there are of each. So they provide only that many paper pieces and pay full postage for those, and a minimal postage for the electronic ones. Not zero -- it is the zero cost that enables spam, after all. With a few cents of cost you still think about the cost of what you are mailing. There is a risk some marketers would want to mail only the electronic customers, and then mail far more stuff since the cost would be a few cents vs. a dollar.

The DMA lobby would probably go nuts fighting this plan, though some of them might love it, since the electronic versions, if looked at, would save a ton of money. And eventually they would just try to get people on "permission marketing" opt-in commercial mail lists, and bypass the postal service and its costs.

So I'm probably dreaming. But it always annoys me to see people generate a big document on a computer and print it on paper for me to toss in the garbage, or at most glance at. The times I would glance, I would be happy enough to get it in electronic form. For those who really want their paper junk mail sometimes, they could offer a service where you click on the junk mail items you liked and they are sent to you on paper later.

New Essay on Autoresponder practices

I wrote earlier this week on the discovery that people were blacklisting sites with email autoresponders. More thought and debate on the issue has led to a number of thoughts over how to solve the issues around autoresponders, in particular the concern that they will respond to messages with forged From addresses.

These thoughts have been laid out in this essay on practices for autoresponders which starts off by pointing to RFC3834, and goes further in a world where people might want to blacklist sites just for autoresponding.

The RFC specfies a way for an autoreponse to be reliabily identified as such. Those who are blacklisting or filtering autoresponders can use this so that if they are going to go about blacklisting a site for running an autoresponder (as is required in the SMTP spec) that they only blacklist further autoresponses, and not ordinary mail from the same server. While some blacklisters, unfortunately, have a capricious disregard for the consequences of their actions, most of them agree that they should wish to block as little legitimate, desired mail as possible, ideally zero, so techniques which can make this happen deserve their attention.

There are many other techniques outlined in my essay on challenge-response best practices which are still not followed (admittedly in a few cases even by my own code, since I never put it into public distribution.) These techniques make C/R not only workable, but I believe a must in any good anti-spam system. If somebody’s anti-spam system is going to block my mail, I want the ability to know about it and reverse that decision by proving I’m not a robot. While it is annoying to have to respond to a challenge, if the alternative is not having your mail read, most people would take the challenge — if it was really necessary. C/R systems allow systems to have no false positives, at least for non-anonymous mailers, and that should be the goal for everybody.

Spamcop blacklists autoresponders

I learned a couple of days ago my mail server got blacklisted by They don’t reveal the reason for it, but it’s likely that I was blacklisted for running an autoresponder, in this case my own custom challenge/response spam filter which is the oldest operating one I know of.

I understand the debate about the merit of C/R spam filters. Like all autoresponders, they can generate unwanted mail when spammers and viruses send mail with a forged From address, and the responder annoys the innocent victim. However, this is a problem common to all autresponders, and unlike the even-more-hated open-relay, it doesn’t magnify the spam problem — there is one possibly annoying response per spam, not hundreds.

I am bothered because I don’t want to see anti-spam advocates fighting other anti-spam methods because they don’t agree with them, or blacklists in general used to punish people you don’t agree with. Spamcop should be fighting spammers, not anti-spammers.

In addition, e-mail autoresponse is an important mail tool. In fact, anti-spammers insist that mailing lists do a confirmed opt-in (also known as double opt-in), generally by autoresponse, before adding a person to a mailing list. When a mail server bounces directly delivered mail it can avoid doing an autoresponse, but if mail comes in through an MX — a vital feature of mail — it requires an autoresponse to bounce it. Vacation programs and many other tools use this ability.

Check to see if your mail system uses as a blacklist. If it does, disable it or switch to something else until they change this policy. Otherwise you won’t receive mail from me, and many others.

Update: My server is no longer blacklisted. I didn’t do anything (other than this blog post and a few complaints to people using the spamcop BL) so perhaps they auto remove. But it could happen again at any time until they change their policy. This is also a nasty DOS attack. Find anybody with any autoresponder, including a bounce of MX’d mail. Send forged mail to it with a From set to a spamtrap address — and they’re blacklisted. Also can be used against any sites that have you enter an E-mail address on a web page and then email that address to confirm you own it — you can get these sites blacklisted trivially. Every web form that can enter an E-mail address is at risk.

Getting the top spammers

A recent item posted on politech and Farber’s IP mailing lists caused some controversy, so I thought I should expand on it here.

The spam law debate has been going on for close to a decade. There are people with many views, and we’ve all heard the other side’s views many times as well. The differences lie in more fundamental values that are hard to change through argument.

Because of that there are giant spam law battles among people who are generally all on the same side — getting rid of spam. Each spam law proposal has people who feel it does too much and chills legitimate speech on one side, and those who feel it does too little and legitimizes some spam on the other. (With many other subtleties as well.)

It’s commonly reported that most spam is sent by a relatively small group of hardcore, heavy volume spammers. In theory much from a group of 20, and the bulk from a group of around 200. I have never known if this is true or not, but a recent conversation with a leading antispam activist gave evidence that it was. Antispammers have tracked down a lot of spam, seen billions of spams come into spam-traps and even infiltrated spammer “bulker” message boards to learn who’s who and how they operate.

So let’s assume for the moment that it’s true that most spam comes from this core group. Let’s focus spam law efforts on a law designed just to get them. A law so narrowly targetted that nobody need fear a chilling effect on legitimate speech, that everybody can get behind. (A law that also makes it clear that it’s not precluding other laws or giving blessing to lesser spammers.)

I would see such a law demanding many criteria. It would require the spammer send millions of spams. It would require the spammer do this with wilful disregard for the consequences — ie. a malicious intent. It could require the spammer have made $10,000 from their spamming. It would also provide funding and direction for law enforcement to actually go after these spammers. It would fine them into bankruptcy (all they ever made from spamming plus punative fines) and possibly jail them, particularly if other criminal actions like fraud, sale of illegal products and computer breakins were involved.

This wouldn’t stop all spammers, but it might well put a real dent in the volume of spam, and scare off many from entering the upper echelons of spamming. This is a great deal more than any other spam law has managed to do.  read more »

More on

In my quest over the leak/sale of the mailing list, I have some amusing updates.

After telling them you don't respond to a "You sold my name" complaint with a request for all of the person's personal information, I got back yet another stock message, "Here's how you can get off our mailing list." I'm getting a lot of companies who use customer service reps for E-mail who clearly never read the E-mails. Yes, I also get software that auto-responds, but amazingly we also get humans who auto-respond.

Anyway, customer service clearly not working, I found their phone number and called their legal dept. where I spoke with a Jill Silverman. She expressed concern after she got clear on what had happened, and asked me to forward her the emails I had gotten to my special address created just for them. I immediately sent them off then heard nothing.

When I inquired again, she told me she never got the E-mails. I figured out why, eventually. The E-mails, which I had put in a text file attachment, were of course spams, and triggered her company's spam filter. Of course, my mail was dropped on the floor, no diagnostic for her or me.

So I put them in a web page and sent her the URL. That should make it through!

Some folks don't get it.

When I give an E-mail address to a web site, I give a different one to each site. I have many domains, including one where all addresses are forwarded to me unless I turn them off.

I recently started getting some spams for "entertainment@" the domain, which is the address I gave to use the web site of, which publishes those books of discount restaurant coupons. Their privacy policy states they don't give away or resell the information, so something's fishy. (And no, since I get all addresses at this domain I know spammers are not dictionary attacking it, this almost certainly leaked or was sold out of

So I file a complaint and get back a form response explaining their privacy policy because I stupidly entered the selectbox saying it was a privacy policy question. When I sent back another note, you will enjoy the response I got back...  read more »

Viagra spam

Thought of the day...

Spam is there to teach us just how many different ways there are to spell Viagra.

Challenge/Response, good or bad

I've just put up a new essay on my web site on whether challenge/response anti-spam systems are good or bad

As some may know, I've been running such a system longer than anybody, having written one in 1997. I wrote a white paper on best practices for such systems that some have found useful.

However, I also see a lot of complaints about C/R systems. Most of those complaints are about the new crop of C/R systems, many of which have annoying bugs. Because some of the concerns are real, however, I felt it was time for an article on those issues for a well-behaved C/R system.

Note that even my own system, in spite of being better behaved that most newer systems, does not meet all my own best practices, though it would if I were writing it again.

Antivirus bounces a curse of their own

I often talk about Challenge Response spam filters because I wrote the first one. One complaint people make is that the filters will challenge even forged mail, causing a challenge to be sent to the forgery victim. While this is not a DOS attack window as some people believe (since you can as easily DOS the target directly as get others to do it for you) it does need more consideration.

However, there are some autoresponders who have no excuse in this, and it is them I am railing on today. With the latest worm program, I am getting "bounces" back from anti-viral mail filters which tell me, "The mail you sent contains a virus and was not delivered."

Of course I didn't send the mail, my address was forged. What bothers me is that the anti-virus program clearly knows there is a virus, and presumably then should know it is the sort of virus which puts in a fake address.

So why it feels the need to send an error to the address it knows is fake, I don't know. The bounces I can tolerate, the bouncing software has no way to know it was a virus, but the anti-virus software has no excuse.

Addon: I'm going to promote a note from the comments because naive me didn't think of it. The virus companies may be happy to send this "your virus was bounced" mail to the wrong address because it's an ad for their anti-virus service.

Stop spam without demanding ID

There's a growing and dangerous movement to try to stop spam by forcing all mail senders to provide ID with each mail they send. Signing mail is not a bad idea, in fact it's quite useful, but to stop spam you have to make everybody sign their mail.

In the past this was a non-starter because this means forcing everybody who mails you to get new mail sending software, or at least to have their ISP do this. But spam has made us so angry people are talking about doing this, even though we don't demand ID for paper mail that, in theory, can contain white powder that can kill you.

This would mean the end to anonymous mail and a lot more complexity in our mail systems. So I sat down and said, if you are ready to force people to get new software, could you stop spam with something more distributed and still allow anonymous mail.

Indeed you could, and I have a proposal outlined to combine CPU stamps, challenge/response and signature to end spam

Syndicate content