Hide full article
A fundamental reason for the proliferation of spam is that the underlying mechanisms for exchanging email over the Internet never check the identity of the sender. Any user anywhere on the Internet can send email that appears to come from any email address. This is a common reason why you receive angry email messages from people asking why you spammed them. You didn't: some spammers used their simple software to spoof your address (usually chosen at random) in spam.
There's a new technique generating some discussion that may change the balance of power and ultimately put more control back in the hands of the owners of individual domain names. It's called Sender Policy Framework (SPF), and it allows a system administrator to tell other mail servers which servers may legitimately send email with a given domain name as the return address.
<http://spf.pobox.com/>
SPF Basics -- The idea behind SPF is simple. Those of us who have domain names, including Internet service providers (ISPs), add records (or have them added on our behalf) that assign IP numbers to domain and host names. For instance, king.tidbits.com currently maps to 216.168.61.154 and emperor.tidbits.com is 216.168.61.78.
These domain records, which are simple text files with one entry per line, also tell mail servers where to deliver mail using mail exchanger (MX) records. The domain record for tidbits.com has an MX record that says to deliver email to emperor.tidbits.com. If that server is busy, an additional entry says to try king.tidbits.com as a backup or secondary.
With SPF, you or a system administrator adds a line that lists the mail servers from which email that is addressed from your domain may be sent. For TidBITS staffers, we would add a line that says, "legitimate email with @tidbits.com in the address must originate from king.tidbits.com or emperor.tidbits.com." Since we often work on the road, we would also say, in SPF format, "or from any SPF mail servers defined by Speakeasy Networks, EarthLink, and Comcast."
Will SPF Work? For SPF to carry out its objective, two things must happen: domain owners have to add SPF records, and mail servers need to be reconfigured to check SPF records before accepting email messages from domains that list SPF records. Both of these are happening simultaneously. AOL, for instance, started listing SPF records weeks ago, and other ISPs may follow. (In fact, SPF was devised by the founder of pobox.com, a popular and long-time email service provider.)
Because there's no penalty in adding SPF records, over 7,500 ISPs (according to the SPF site) have already added them. The SPF site offers a wizard for composing these records to avoid learning the syntax by hand. For instance, the SPF record for my glennf.com domain will look like this:
"v=spf1 a mx ptr ip4:64.81.13.192/26 include:speakeasy.net -all"
<http://spf.pobox.com/wizard.html>
For many users, composing these entries will still be too technical. They can look for assistance to their domain hosts; the company TidBITS relies on, easyDNS, has already indicated to us that they're working on supporting SPF. When a domain hosting company supports SPF, it should be even simpler for users to add SPF settings.
The other half requires more effort. The SPF site lists patches and beta test versions for some major mail transfer agents (MTAs), the formal name for mail servers that receive messages addressed to users at any domain for which the server accepts email. This includes Postfix (the default mail server in Mac OS X 10.3), Sendmail (which is widely used throughout the Unix and Linux world and which Apple included with Mac OS X 10.2 and earlier), Exim, and Qmail.
<http://spf.pobox.com/downloads.html>
SpamAssassin 2.70 will also include SPF support as part of its scoring system.
Flies in the Ointment: Legitimate Spoofing -- As email and anti-spam consultant John Levine pointed out to Adam Engst and me via email and in an essay he's posted, the fundamental problem that SPF is solving isn't precisely spam, but spoofing, and it's not at all uncommon to rely on systems that operate by spoofing mail legitimately.
<http://www.taugh.com/mp/lmap.html>
Mailing lists are the most problematic, since if someone sends mail to certain discussion lists, the message as sent by the mailing list will appear to be from that person, but will be sent out via the list server. Assuming the poster isn't using an account at the same domain as the list server, any SPF checks would fail, since the list server wouldn't be SPF-approved for mail from the sender's domain.
Also, many sites let you forward an article to someone else. These services typically require you to enter your return address and the publication spoofs your address so the message appears to come from you and so any replies go back to you.
Finally, many email forwarding services rewrite incoming message headers so forwarded messages look like they came from the original sender instead of the forwarding service. I use my alumni association's free forwarding from aya.yale.edu to my address, and many other mailing services allow the same kind of forwarding.
The mailing list and forwarding problems would require reworking of many aspects of email systems. The end result might be better, but the transition could be painful. John Levine offered some suggestions in his essay above for plastering over the problem, and the SPF site has specific suggestions as well.
<http://spf.pobox.com/faq.html#forwarding>
These three spoof-at-your-request problems remain the biggest obstacles facing broad SPF adoption.
Isn't Microsoft Already Doing This? Microsoft made some interesting announcements a few weeks ago at the RSA security conference about a strategy similar to SPF called Caller ID.
<http://www.pcworld.com/news/article/ 0,aid,114911,00.asp>
<http://www.microsoft.com/mscorp/twc/privacy/ spam_callerid.mspx>
Caller ID must download and parse an entire email message before it can apply itself, and it uses XML (Extensible Markup Language) to encode information in the DNS record. SPF is less formal (although its proponents are working towards having it ratified as an Internet standard) but it can work with just the message envelope, which mail servers read before they even see the message headers and body.
SPF-enabled mail servers should be able to read Caller ID records, however, because of the similarity in approach. They're not inconsistent with one another, despite their different tacks.
Caller ID might better avoid issues with the kinds of legitimate spoofing described above because it may be capable of better analyzing the path of legal forwarded addresses or mailing list addresses.
From what I can tell at the moment, Microsoft will provide royalty-free licenses for any patents necessary to implement Caller ID. There are currently no patents associated with SPF, and its inventor has publicly declared his intent to keep SPF royalty-free should any future defensive patents become necessary.
Can SPF Succeed? SPF definitely suffers from the chicken and egg problem, but early adoption from AOL and other major ISPs might give it a boost. I expect that many large ISPs will find it worthwhile to adopt SPF as soon as the mail server software is widely tested - no ISP wants to be an early adopter - because it could radically reduce the amount of email that they process and store.
Early adoption by ISPs has a huge advantage: the ISPs could start preventing the massive amount of returned email that wasn't sent from their users, which is part of the overwhelming problem of spam - but only part of it.
If SPF is adopted by a large number of ISPs, spammers will start using domains that lack SPF records, and it's likely that those domains would be shunned, much as happens with domains that allow open relay exploitation by spammers. Because most domains are hosted by ISPs, the ISPs would then encourage or even require their customers with domains to have SPF records.
Ultimately, as the SPF site itself notes, spammers would register new domains and provide SPF records for those domains, but ISPs could out-evolve this approach given that existing tools could easily recognize and block email from domains - particularly newly registered domains - that send only spam.
Spammers Always Find a Way -- Of course, spammers always find a way around any difficulty. It's hard to blame them in one sense, because spam is evolution in action. Much like fruit flies cycle through generations so quickly they're used to test ideas of genetics, spammers send out so many billions of messages that it's natural (in the worst sense) that the ones that slip through inform them how to build better spam-sending engines. This doesn't mean we in any way approve of spam, but it does explain the inevitability of spam adaptations.
SPF solves a part of the problem, but not the whole problem. Spoofing is just one aspect of spam, but reducing even part of the spoofing problem reduces the overall demands on each email system as well as the amount of illegitimate email that's sent. No magic bullet exists that will end the battle against spam, but all domain owners should take a hard look at SPF. It's just one more tool in the arsenal of keeping a clean mailbox, but either SPF or something like it is in our future.
