The Tlog – a technology blog

To end my SPF series, I’m going to consider the following question: what if everyone used SPF? Would it “end” spam?

The answer is, of course, “no, but…”. But, first, let’s understand the question itself.

In “everyone used”, what does “used” mean? If you followed my previous posts on this subject, you’ll know that there are two distinct parts: having an SPF record for your domain, and configuring your SMTP server to reject email purporting to be from an address with a valid SPF record, when it doesn’t come from an authorized server.

Let’s assume that the question implies both.

So, if every legit organization had an SPF record and enforced SPF in their incoming email servers, what would it mean?

It still depends on what was meant by “enforced”. For instance, as it is now, it makes sense to use an SPF record (and reject mail coming from an unauthorized server, as I mentioned before), but not to require an SPF record, as most of the world is still not using it.

If that changed, though… it would certainly make things a lot easier for the “good guys”. Think about it: what does SPF prevent? The faking of sender addresses. Who ever does that? Spammers. Therefore, who has ever a reason not to use an SPF record? Spammers. In a world where every legitimate organization used SPF, having such a record wouldn’t mean that the sender wasn’t a spammer, but not having one would certainly mean that he was. Ergo, reject any mail from a domain without an SPF record, even before verifying whether the origin server is authorized for that domain.

Of course, spammers would adapt, and have SPF records for their own domains. But never again would they be able to fake a sender address. They would never again be able to efficiently pretend to be your internet provider, or your bank, or Facebook, or anything like that. They would have to use their own domains in the sender address… and they don’t exactly tend to look “nice”; besides, they’re not what your ISP or your bank would use.

A world with SPF would mean a world where you could actually trust the “From:” field. Can you imagine such a thing?

In the previous post, I wrote, at the end of the recipe:

restart Postfix. Check your logs to see if everything is working properly.

But what to look for? Here are a few real life examples, from my own Postfix log file:

Nov 26 07:07:27 sh postfix/policy-spf[10685]: : SPF none (No applicable sender policy available): Envelope-from: bounce-1492514-406278644@lyris.diskeepermail.com
Nov 26 07:07:27 sh postfix/policy-spf[10685]: handler sender_policy_framework: is decisive.
Nov 26 07:07:27 sh postfix/policy-spf[10685]: : Policy action=PREPEND Received-SPF: none (lyris.diskeepermail.com: No applicable sender policy available) receiver=sh.dehumanizer.com; identity=mailfrom; envelope-from="bounce-1492514-406278644@lyris.diskeepermail.com"; helo=lyris.diskeepermail.com; client-ip=68.177.217.241

This (note the “SPF none”) is what will happen for most emails: no SPF record exists. Unfortunately, most of the world is still not using it. The mail passes this check, since it’s not a good idea to enforce SPF yet (but may fail one of the remaining ones, such as an RBL list or SpamAssassin.)

Nov 24 16:19:56 sh postfix/policy-spf[32619]: : SPF pass (Mechanism 'ip4:209.128.72.240/28' matched): Envelope-from: promotions@iambic.com
Nov 24 16:19:56 sh postfix/policy-spf[32619]: handler sender_policy_framework: is decisive.
Nov 24 16:19:56 sh postfix/policy-spf[32619]: : Policy action=PREPEND Received-SPF: pass (iambic.com: 209.128.72.242 is authorized to use 'promotions@iambic.com' in 'mfrom' identity (mechanism 'ip4:209.128.72.240/28' matched)) receiver=sh.dehumanizer.com; identity=mailfrom; envelope-from="promotions@iambic.com"; helo=ns1.iambic.com; client-ip=209.128.72.242

Above is an example of a legitimate mail from a sender with a properly configured SPF record. The record says “here are the server(s) that send email from this domain”, and the origin server is indeed one of those. The email passes (note the “SPF pass”), and, if you’re using SpamAssassin, it takes that into account, meaning that the mail is less likely to be considered spam. (See? That’s a great reason to have an SPF record for your domain!)

Finally, the following:

Nov 24 19:57:13 sh postfix/policy-spf[1210]: : SPF fail (Mechanism '-all' matched): Envelope-from: pedro@dehumanizer.com
Nov 24 19:57:13 sh postfix/policy-spf[1210]: handler sender_policy_framework: is decisive.
Nov 24 19:57:13 sh postfix/policy-spf[1210]: : Policy action=550 Please see http://www.openspf.org/Why?s=mfrom;id=pedro%40dehumanizer.com;ip=204.212.122.254;r=sh.dehumanizer.com

is a good demonstration of the usefulness of SPF. You’ll notice that the mail pretends to come from my own domain, but since I have configured the SPF record for it, Postfix is able to see that the origin server is not authorized to send mail from that domain. And, since my record ends with “-all“, it means that the record is to be “taken seriously”, so the mail is refused then and there (note the “SPF fail”), even before checking RBL lists and the like.

(for extra fun, read parts 1 and 2 first.)

Now that you know how to configure an SPF record for your domain(s), the natural next step, if you administer an email server, is to start checking SPF records for mails you receive.

Now, spammers are infamous for not respecting rules, not “playing nice”, so, you might ask, what makes me think they’ll set up SPF records for their domains, which would be kind of self-defeating? The obvious answer is that SPF doesn’t depend on the spammers’ collaboration. Since legitimate email senders use SPF to tell the world which servers they use to send email, it prevents others — such as the aforementioned spammers — from faking sender addresses to pretend they’re from those senders / domains.

To put it simply: if there’s an SPF record for the “gmail.com” domain, then you can — and should — reject mail purporting to be from something@gmail.com that doesn’t come from the servers listed in that record. In other words, anyone who fakes a @gmail.com address can’t fool your server, assuming you have it configured to use SPF.

Now, how to do that? There are many ways, of course, depending on the email server you use. The simple recipe below uses postfix-policyd-spf-perl to make Postfix reject mail from domains with properly configured SPF records, when the mail comes from an unauthorized server (that is, one not listed on the record.) This assumes you already have Postfix up and running.

• install postfix-policyd-spf-perl. In Ubuntu, just do an apt-get install postfix-policyd-spf-perl .
• add this line to /etc/postfix/main.cf:
  spf-policyd_time_limit = 3600s
• add the following to /etc/postfix/master.cf:

  policy-spf  unix  -       n       n       -       -       spawn
       user=nobody argv=/usr/bin/policyd-spf

  (change the path of policyd-spf if it’s installed somewhere else; that’s where the Ubuntu package puts it.)

• in /etc/postfix/main.cf, find the smtpd_recipient_restrictions section, and, immediately after permit_mynetworks (and permit_sasl_authenticated, if you’re using that), add:
  check_policy_service unix:private/policy-spf
• restart Postfix. Check your logs to see if everything is working properly.

Now, remember when in part 2 I mentioned a lesser-known reason for configuring the SPF record for your domain? It’s this: it’ll stop a lot of incoming spam. These days, a lot of spam email pretends to be from you (that is, it uses your email address as both the “From:” and “To:”), or at least from your domain (e.g. administrator@yourdomain, manager@yourdomain, and so on.) I don’t know why spammers do that, but apparently it works, or else they wouldn’t do it (maybe victims get confused and think the email comes from someone in their company, or maybe webmail services tend — or at least did so once — to trust the user’s own address, I don’t know.) Well, if you have SPF for your domain and your email server checks SPF records for incoming mail, then spam messages such as these will be always rejected, even when other methods of stopping spam (RBL lists, SpamAssassin, etc.) fail. Never again will you receive emails pretending to be from your own address.

(this is part 2 of a series. You should read part 1 first, and after this post you should read part 3.)

Suppose you have a domain and send email from it (in fact, even if you don’t, this is still a good idea; more about that later). How to configure SPF for it?

Easy: you simply add a specially formatted TXT record to the domain. (Where? If you have your own public DNS servers, you’ll have to edit the zone file¹; else, if your domain (note: the domain, not the web server) is maintained by a registrar or ISP, it should provide for you an administration interface where you are able to add and edit records.)

OpenSPF.org provides a “wizard” to build an SPF TXT record for your domain, and I suggest you try it out. However, I think I can give you an example that is mostly self-explanatory, which is the SPF TXT record for my own dehumanizer.com domain:

v=spf1 a a:mail.dehumanizer.com a:sh.dehumanizer.com a:sh2.dehumanizer.com mx include:netcabo.pt -all

(In fact, that record is kind of overkill; I could remove about half of it and everything would still work. But it’s useful as an example.)

v=spf1 means it’s an SPF record.

a means that “dehumanizer.com” (the host that that name resolves to) is authorized for that domain. I could remove that one, as the following one takes care of it (it’s the same host).

a:mail.dehumanizer.com means that mail.dehumanizer.com is authorized. The same for the following two (sh.dehumanizer.com and sh2.dehumanizer.com; the latter doesn’t exist anymore, so I could remove it.)

mx means that the mx records for the domain are also authorized. They’re currently mail.dehumanizer.com and sh.dehumanizer.com, which were previously taken care of, so I could remove this one as well.

include:netcabo.pt means that whatever is authorized for the netcabo.pt domain is also authorized here. This is from when I used my ISP as a smart host, some time ago; as I no longer do that, I could remove this one too.

Finally, -all means, in effect, that, yes, I’m serious about this, all of the authorized servers for this domain are listed, which means that any other hosts are unauthorized, and emails coming from them with a “From:” of this domain should be refused. You should configure this part only after successfully testing your configuration. Until then, there’s a “soft fail” option, ~all, which means the same, but adds “but don’t take it seriously.”

If you’ve been paying attention, you’ve probably already noticed that I could replace that record with the much simpler:

v=spf1 a:mail.dehumanizer.com a:sh.dehumanizer.com -all

for the exact same results (and I’ll probably do that a bit later today.)

What’s the benefit of this? Well, as you are telling the world “mails from my domain come only from my servers”, the rest of the world will more easily be able to kill mails pretending to be from your domain… while also “respecting” legit mail more. In effect, you tell other servers “here’s how you can tell which mails from my domain are genuine”. Many ISPs and filtering systems (such as SpamAssassin) take advantage of a properly configured SPF record to raise the “trustworthiness” of emails coming from the correct servers (while lowering it, or even simply refusing to accept the mail, for messages coming from a server not listed in the SPF record.)

There’s also another (lesser-known) advantage of using SPF for your domain, when you also receive email to it. But that’s for part 3, where I’ll explain how to configure Postfix to look at SPF records when receiving mail…

1. note: put the entire record contents between quotes; e.g. for Bind, use dehumanizer.com. IN TXT "v=spf1 a a:mail.dehumanizer.com a:sh.dehumanizer.com a:sh2.dehumanizer.com mx include:netcabo.pt -all" (that’s a single line) [↩]

(Note: this is the first of a series of posts related to email servers and spam. This one is more of a theoretical intro; future posts will delve into the gory details.)

(Later note: here are parts 2 and 3.)

You may have heard of SPF (Sender Policy Framework), but what is it? To put it simply, it’s a way for domain owners to say to the world: “these servers are the only ones that send email from this domain.”

What is the point? Well, if you have been using email for any amount of time, you’ve probably noticed that a lot of spam fakes its sender address (the “From:” field). In fact, you may have found yourself that it’s incredibly easy to do; most SMTP servers simply accept any sender, as long as either 1) the destination address is theirs, or 2) your IP address is on their client list. In other words, a company’s email server accepts mail to the company’s employees, and also allows those employees to use it to send mail to the rest of the world.

SPF, when correctly configured by everyone involved, prevents that.

“Everyone involved” is, of course, an utopic scenario. Still, there is no reason for you not to do your part, whether you are a domain owner who sends email from that domain, the administrator of an email server that receives mail to its users, or both.

More to come…