Email is hard.

The standards we use for email date back to the 1980s. They were based on even more primitive email standards develiped in the late 1960s and early 1970s.

Computer networks were a very different animal back then. The ARPAnet, one of the precursors to the modern Internet, had 50 systems on it. Everyone knew each other. Only a small handful of “email addresses” existed. There was no security and no authentication, because you knew all the other people who had email access.

Today’s email system is a hacked-together, tottering patchwork of different ideas and implementations, with all kinds of additions and extensions bolted on. It’s still woefully insecure, and it still has its roots in an earlier and vastly simpler time.

This means running email servers is hard. Even if you’re a big ISP, running email servers is hard. And it’s expensive. Even the most dedicated sendmail guru will tell you getting all the configuration wibbly bits correct is difficult and tedious, and it’s easy to make mistakes.

So more and more people are outsourcing their email. Even large ISPs are turning to Google to run their mail servers. Everyone knows about gmail, but most people don’t know that gmail can also take over your company’s mail services, dropping the “@gmail” bit for whatever you want. Google is good at email and it’s a lot cheaper to have them run your email than it is to do it yourself.

Which creates a problem.

---

Most email is spam, by a huge margin. About three-quarters of all the email sent anywhere is spam. The only reason you can still use your email is filtering, filtering, filtering. The stuff that lands in your inbox is the tiny drip, drip, drip of spam that gets through the filters holding back the torrential flood.

This happens because email standards were invented in a time when there were 50 computers on the entire net and everyone knew everyone else, so there is absolutely no authentication built into email. I can send you mail from any address I want and your server will blindly accept it.

Now, most of the Internet doesn’t like spam. Or, at least, it pretends not to. (Many mainstream ISPs and affiliate advertising companies turn a blind eye to it, because profit–but that’s a post I’m working on for another day.)

ISPs have certain “role accounts”–email adddresses that are always the same, such as postmaster@whatever, hostmaster@whatever, and abuse@whatever.

The abuse@ email address is where you send reports of, naturally, abuse. If an ISP is hosting a Spamvertised Web site, or has been hacked and is being used to spread viruses, or is the source of spam emails, you send notifications and copies of the spam emails to abuse@.

So, naturally, you can’t put spam filters on the abuse@ email address, for obvious reasons. If you spam-filter abuse@ and I try to send you notification of spam that’s being sent from your servers, the notification will get filtered and you won’t see it.

In fact, “thou shalt not put spam filters on your abuse role account” is in one of the documents that specifies what makes the Internet go. The standards and protocols that make the Internet work are outlined in a series of technical documents called “RFC”s, and RFC2142 spells out what role accounts an ISP should have, what they’re used for…and oh yeah, don’t run a spam filter on your abuse@ address because that would be really stupid.

The problem is that more and more ISPs are realizing that email is hard, running email servers is hard, and it’s a lot cheaper and easier to let Google just handle all your email services for you.

And Google automatically filters spam.

---

Email is hard.

Part of the reason email is hard is every email address can be configured in a zillion different ways with a zillion different options.

Google has built a set of options that make sense for most email addresses most of the time, and when you turn over your email operations to Google, that’s what you get.

One of those options that makes sense for most email addresses most of the time is spam filtering. When ISPs and Web service providers relinquish control of their email services to Google, they’re often not even aware that Google filters spam by default. They don’t know they are filtering their abuse@ address, because who would do that? How dumb would you have to be to put a spam filter on an email address intended for reporting spam, right?

So we get things like this:

Here’s the bounce:

: host aspmx.l.google.com[173.194.64.27] said: 550-5.7.1
[67.18.53.18 7] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
Gmail, 550-5.7.1 this message has been blocked. Please visit 550-5.7.1
http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550
5.7.1 more information. ny4si6062371obb.164 – gsmtp (in reply to end of
DATA command)
Reporting-MTA: dns; gateway07.websitewelcome.com
X-Postfix-Queue-ID: 0FF09169EDAB
X-Postfix-Sender: rfc822; franklin@franklinveaux.com
Arrival-Date: Fri, 28 Mar 2014 16:31:17 -0500 (CDT)

This was a bounce that came back from a “phish”–a phony PayPal or bank site designed to trick people into giving up sensitive information–that Cloudflare, a content delivery network, was serving. I reported the phish to them on March 28. When I checked it three days ago, it was still there, still stealing people’s passwords.

And it’s not isolated. This is an incredibly common problem:

: host alt2.ASPMX.L.GOOGLE.com[74.125.29.27] said:
550-5.7.1 [67.18.62.19 12] Our system has detected that this message
is 550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
Gmail, 550-5.7.1 this message has been blocked. Please visit 550-5.7.1
http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550
5.7.1 more information. x7si1316702qaj.209 – gsmtp (in reply to end of DATA
command)
Reporting-MTA: dns; gateway01.websitewelcome.com
X-Postfix-Queue-ID: C61B24C69D52
X-Postfix-Sender: rfc822; franklin@franklinveaux.com
Arrival-Date: Sat, 3 May 2014 15:54:51 -0500 (CDT)

: host ASPMX.L.GOOGLE.com[173.194.64.27] said: 550-5.7.1
[67.18.22.93 12] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to
Gmail, 550-5.7.1 this message has been blocked. Please visit 550-5.7.1
http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550
5.7.1 more information. ij7si5132986obc.180 – gsmtp (in reply to end of
DATA command)
Reporting-MTA: dns; gateway05.websitewelcome.com
X-Postfix-Queue-ID: 9FB7A4A9184F7
X-Postfix-Sender: rfc822; franklin@franklinveaux.com
Arrival-Date: Mon, 5 May 2014 02:07:56 -0500 (CDT)

Most folks, when they see the bounce message, are like “d’oh!” and find a way to turn off filtering their abuse@ message. (Cloudflare seems to be a bit of a special case; they tend to get defensive and snarky instead. That’s disappointing, as their founder was an early anti-spam pioneer.)

The dangers of outsourcing bits of your business is that you necessarily lose control of those bits. When you’re an ISP or a Web service provider and you outsource your email services, well, losing control of your email services can have some unfortunate consequences. When you filter your abuse@ address, you soon become a haven for spam and malware and phish pages and all sorts of other nasties…because you don’t know you’re hosting them.

So what’s the solution?

Ideally, a complete overhaul of email. Since that’s about as likely as Elvis stepping out of a flying saucer in Times Square and handing me a winning Powerball lotto ticket, I’m not holding my breath.

Another solution is for ISPs to acknowledge that the work they do is hard, and just doing it. That’s a bit more likely, but it still involves things approximately as probable as Elvis and flying saucers–perhaps Elvis handing me a chocolate bagel rather than a Powerball ticket–so I’m still not holding my breath.

But it might be in the realm of possibility for Google to set up their configuration to turn off spam filtering by default on any email address that contains the word “abuse.”

Anyone know anyone who works in Google’s email services department?