To end my SPF series, I’m going to consider the following question: what if everyone used SPF? Would it “end” spam?

The answer is, of course, “no, but…”. But, first, let’s understand the question itself.

In “everyone used”, what does “used” mean? If you followed my previous posts on this subject, you’ll know that there are two distinct parts: having an SPF record for your domain, and configuring your SMTP server to reject email purporting to be from an address with a valid SPF record, when it doesn’t come from an authorized server.

Let’s assume that the question implies both.

So, if every legit organization had an SPF record and enforced SPF in their incoming email servers, what would it mean?

It still depends on what was meant by “enforced”. For instance, as it is now, it makes sense to use an SPF record (and reject mail coming from an unauthorized server, as I mentioned before), but not to require an SPF record, as most of the world is still not using it.

If that changed, though… it would certainly make things a lot easier for the “good guys”. Think about it: what does SPF prevent? The faking of sender addresses. Who ever does that? Spammers. Therefore, who has ever a reason not to use an SPF record? Spammers. In a world where every legitimate organization used SPF, having such a record wouldn’t mean that the sender wasn’t a spammer, but not having one would certainly mean that he was. Ergo, reject any mail from a domain without an SPF record, even before verifying whether the origin server is authorized for that domain.

Of course, spammers would adapt, and have SPF records for their own domains. But never again would they be able to fake a sender address. They would never again be able to efficiently pretend to be your internet provider, or your bank, or Facebook, or anything like that. They would have to use their own domains in the sender address… and they don’t exactly tend to look “nice”; besides, they’re not what your ISP or your bank would use.

A world with SPF would mean a world where you could actually trust the “From:” field. Can you imagine such a thing?

In the previous post, I wrote, at the end of the recipe:

restart Postfix. Check your logs to see if everything is working properly.

But what to look for? Here are a few real life examples, from my own Postfix log file:

Nov 26 07:07:27 sh postfix/policy-spf[10685]: : SPF none (No applicable sender policy available): Envelope-from: bounce-1492514-406278644@lyris.diskeepermail.com
Nov 26 07:07:27 sh postfix/policy-spf[10685]: handler sender_policy_framework: is decisive.
Nov 26 07:07:27 sh postfix/policy-spf[10685]: : Policy action=PREPEND Received-SPF: none (lyris.diskeepermail.com: No applicable sender policy available) receiver=sh.dehumanizer.com; identity=mailfrom; envelope-from="bounce-1492514-406278644@lyris.diskeepermail.com"; helo=lyris.diskeepermail.com; client-ip=68.177.217.241

This (note the “SPF none”) is what will happen for most emails: no SPF record exists. Unfortunately, most of the world is still not using it. The mail passes this check, since it’s not a good idea to enforce SPF yet (but may fail one of the remaining ones, such as an RBL list or SpamAssassin.)

Nov 24 16:19:56 sh postfix/policy-spf[32619]: : SPF pass (Mechanism 'ip4:209.128.72.240/28' matched): Envelope-from: promotions@iambic.com
Nov 24 16:19:56 sh postfix/policy-spf[32619]: handler sender_policy_framework: is decisive.
Nov 24 16:19:56 sh postfix/policy-spf[32619]: : Policy action=PREPEND Received-SPF: pass (iambic.com: 209.128.72.242 is authorized to use 'promotions@iambic.com' in 'mfrom' identity (mechanism 'ip4:209.128.72.240/28' matched)) receiver=sh.dehumanizer.com; identity=mailfrom; envelope-from="promotions@iambic.com"; helo=ns1.iambic.com; client-ip=209.128.72.242

Above is an example of a legitimate mail from a sender with a properly configured SPF record. The record says “here are the server(s) that send email from this domain”, and the origin server is indeed one of those. The email passes (note the “SPF pass”), and, if you’re using SpamAssassin, it takes that into account, meaning that the mail is less likely to be considered spam. (See? That’s a great reason to have an SPF record for your domain!)

Finally, the following:

Nov 24 19:57:13 sh postfix/policy-spf[1210]: : SPF fail (Mechanism '-all' matched): Envelope-from: pedro@dehumanizer.com
Nov 24 19:57:13 sh postfix/policy-spf[1210]: handler sender_policy_framework: is decisive.
Nov 24 19:57:13 sh postfix/policy-spf[1210]: : Policy action=550 Please see http://www.openspf.org/Why?s=mfrom;id=pedro%40dehumanizer.com;ip=204.212.122.254;r=sh.dehumanizer.com

is a good demonstration of the usefulness of SPF. You’ll notice that the mail pretends to come from my own domain, but since I have configured the SPF record for it, Postfix is able to see that the origin server is not authorized to send mail from that domain. And, since my record ends with “-all“, it means that the record is to be “taken seriously”, so the mail is refused then and there (note the “SPF fail”), even before checking RBL lists and the like.

(for extra fun, read parts 1 and 2 first.)

Now that you know how to configure an SPF record for your domain(s), the natural next step, if you administer an email server, is to start checking SPF records for mails you receive.

Now, spammers are infamous for not respecting rules, not “playing nice”, so, you might ask, what makes me think they’ll set up SPF records for their domains, which would be kind of self-defeating? The obvious answer is that SPF doesn’t depend on the spammers’ collaboration. Since legitimate email senders use SPF to tell the world which servers they use to send email, it prevents others — such as the aforementioned spammers — from faking sender addresses to pretend they’re from those senders / domains.

To put it simply: if there’s an SPF record for the “gmail.com” domain, then you can — and should — reject mail purporting to be from something@gmail.com that doesn’t come from the servers listed in that record. In other words, anyone who fakes a @gmail.com address can’t fool your server, assuming you have it configured to use SPF.

Now, how to do that? There are many ways, of course, depending on the email server you use. The simple recipe below uses postfix-policyd-spf-perl to make Postfix reject mail from domains with properly configured SPF records, when the mail comes from an unauthorized server (that is, one not listed on the record.) This assumes you already have Postfix up and running.

• install postfix-policyd-spf-perl. In Ubuntu, just do an apt-get install postfix-policyd-spf-perl .
• add this line to /etc/postfix/main.cf:
  spf-policyd_time_limit = 3600s
• add the following to /etc/postfix/master.cf:

  policy-spf  unix  -       n       n       -       -       spawn
       user=nobody argv=/usr/bin/policyd-spf

  (change the path of policyd-spf if it’s installed somewhere else; that’s where the Ubuntu package puts it.)

• in /etc/postfix/main.cf, find the smtpd_recipient_restrictions section, and, immediately after permit_mynetworks (and permit_sasl_authenticated, if you’re using that), add:
  check_policy_service unix:private/policy-spf
• restart Postfix. Check your logs to see if everything is working properly.

Now, remember when in part 2 I mentioned a lesser-known reason for configuring the SPF record for your domain? It’s this: it’ll stop a lot of incoming spam. These days, a lot of spam email pretends to be from you (that is, it uses your email address as both the “From:” and “To:”), or at least from your domain (e.g. administrator@yourdomain, manager@yourdomain, and so on.) I don’t know why spammers do that, but apparently it works, or else they wouldn’t do it (maybe victims get confused and think the email comes from someone in their company, or maybe webmail services tend — or at least did so once — to trust the user’s own address, I don’t know.) Well, if you have SPF for your domain and your email server checks SPF records for incoming mail, then spam messages such as these will be always rejected, even when other methods of stopping spam (RBL lists, SpamAssassin, etc.) fail. Never again will you receive emails pretending to be from your own address.

As an intermission (there’s more to come in the SPF series), here’s what’s changed on my server since, oh, about a year and a half ago:

• The OS is now Ubuntu Karmic Koala (9.10), and all the server’s running software comes from the standard Ubuntu packages, which means that whatever version is in Karmic¹, that’s what I’m running here.
• I’ve switched, definitely, from Apache to nginx. It’s faster, more efficient, and ridiculously easier to configure (to put it in perspective, in terms of ease of configuration, it’s like OpenBSD’s amazing pf to the unholy abomination that is Linux’s iptables.) nginx works perfectly with the two pieces of software I use the most on my server, WordPress and MyBB, even while using a SEO plugin with the latter, which requires some non-trivial redirect rules.
• As I don’t trust any ISP’s email server to distribute the mail my server sends (mostly confirmation emails from my forums), and since most ISPs and companies these days block mail sent from dynamic IP addresses, I keep a 256 MB Slicehost slice, with a static IP address and, most importantly, reverse DNS, which I use as a smart host for my home server. The slice doesn’t have a lot of power in terms of CPU (and it’s not meant to), but, as bandwidth is much cheaper in the US than in my poor country, the “small” limit the cheapest slice includes is a lot; I use it for serving static files, mostly for my forums (all images and Javascript files are served from there), and I still have bandwidth to spare.
• I’ve stopped using a Squid proxy in my home network, and nowadays access the web directly… except for when I indulge in one of my newest weird habits: reading webcomics like this one or this one while having lunch or dinner. At such a time, moving instantly from comic to comic is a must… so I simply re-enable Squid (with more aggressive caching than I’d use for normal browsing; after all, existing comics typically aren’t going to change, are they?), do a nice little wget in my server to download and cache the entire comic, and then enjoy reading the whole of it (in as many meals as it takes) as if it was stored locally…
• A few changes to my email server’s configuration, mostly related to spam filtering… but I’ve been writing about that, haven’t I? And there’s still more to come.

1. with updates, of course — people who are afraid of installing updates (“but… it might break something!”) are nothing more than mewling weaklings who are utterly incompetent as sysadmins; they should never be allowed within a mile of any server. Even one running Windows.

(this is part 2 of a series. You should read part 1 first, and after this post you should read part 3.)

Suppose you have a domain and send email from it (in fact, even if you don’t, this is still a good idea; more about that later). How to configure SPF for it?

Easy: you simply add a specially formatted TXT record to the domain. (Where? If you have your own public DNS servers, you’ll have to edit the zone file¹; else, if your domain (note: the domain, not the web server) is maintained by a registrar or ISP, it should provide for you an administration interface where you are able to add and edit records.)

OpenSPF.org provides a “wizard” to build an SPF TXT record for your domain, and I suggest you try it out. However, I think I can give you an example that is mostly self-explanatory, which is the SPF TXT record for my own dehumanizer.com domain:

v=spf1 a a:mail.dehumanizer.com a:sh.dehumanizer.com a:sh2.dehumanizer.com mx include:netcabo.pt -all

(In fact, that record is kind of overkill; I could remove about half of it and everything would still work. But it’s useful as an example.)

v=spf1 means it’s an SPF record.

a means that “dehumanizer.com” (the host that that name resolves to) is authorized for that domain. I could remove that one, as the following one takes care of it (it’s the same host).

a:mail.dehumanizer.com means that mail.dehumanizer.com is authorized. The same for the following two (sh.dehumanizer.com and sh2.dehumanizer.com; the latter doesn’t exist anymore, so I could remove it.)

mx means that the mx records for the domain are also authorized. They’re currently mail.dehumanizer.com and sh.dehumanizer.com, which were previously taken care of, so I could remove this one as well.

include:netcabo.pt means that whatever is authorized for the netcabo.pt domain is also authorized here. This is from when I used my ISP as a smart host, some time ago; as I no longer do that, I could remove this one too.

Finally, -all means, in effect, that, yes, I’m serious about this, all of the authorized servers for this domain are listed, which means that any other hosts are unauthorized, and emails coming from them with a “From:” of this domain should be refused. You should configure this part only after successfully testing your configuration. Until then, there’s a “soft fail” option, ~all, which means the same, but adds “but don’t take it seriously.”

If you’ve been paying attention, you’ve probably already noticed that I could replace that record with the much simpler:

v=spf1 a:mail.dehumanizer.com a:sh.dehumanizer.com -all

for the exact same results (and I’ll probably do that a bit later today.)

What’s the benefit of this? Well, as you are telling the world “mails from my domain come only from my servers”, the rest of the world will more easily be able to kill mails pretending to be from your domain… while also “respecting” legit mail more. In effect, you tell other servers “here’s how you can tell which mails from my domain are genuine”. Many ISPs and filtering systems (such as SpamAssassin) take advantage of a properly configured SPF record to raise the “trustworthiness” of emails coming from the correct servers (while lowering it, or even simply refusing to accept the mail, for messages coming from a server not listed in the SPF record.)

There’s also another (lesser-known) advantage of using SPF for your domain, when you also receive email to it. But that’s for part 3, where I’ll explain how to configure Postfix to look at SPF records when receiving mail…

1. note: put the entire record contents between quotes; e.g. for Bind, use dehumanizer.com. IN TXT "v=spf1 a a:mail.dehumanizer.com a:sh.dehumanizer.com a:sh2.dehumanizer.com mx include:netcabo.pt -all" (that’s a single line)

(Note: this is the first of a series of posts related to email servers and spam. This one is more of a theoretical intro; future posts will delve into the gory details.)

(Later note: here are parts 2 and 3.)

You may have heard of SPF (Sender Policy Framework), but what is it? To put it simply, it’s a way for domain owners to say to the world: “these servers are the only ones that send email from this domain.”

What is the point? Well, if you have been using email for any amount of time, you’ve probably noticed that a lot of spam fakes its sender address (the “From:” field). In fact, you may have found yourself that it’s incredibly easy to do; most SMTP servers simply accept any sender, as long as either 1) the destination address is theirs, or 2) your IP address is on their client list. In other words, a company’s email server accepts mail to the company’s employees, and also allows those employees to use it to send mail to the rest of the world.

SPF, when correctly configured by everyone involved, prevents that.

“Everyone involved” is, of course, an utopic scenario. Still, there is no reason for you not to do your part, whether you are a domain owner who sends email from that domain, the administrator of an email server that receives mail to its users, or both.

More to come…

Now that I work at home (and yes, I’ve been incredibly lazy… where are the new posts? ahem… any day now ), I’ve discovered something about myself: I don’t like to read stuff on my computer.

Sure, I do a lot of that, anyway, but, for instance, there are a lot of blogs that I have subscribed in Bloglines… only I usually skip them. It’s not that I don’t enjoy the content of those blogs – I do. I simply tend, repeatedly, to find excuses, other things I’d rather do at the time, and so on. However, I love to read them in other places – in bed, in a café, and so on. Whenever I’m alone, with nothing to do, and away from a computer.

Reading them in a mobile phone or PDA, then, is the logical answer (a laptop is still much too “PC-like”, with all its myriad distractions; when you can do everything, sometimes it’s hard to focus on just what you need to actually do). I currently use a Nokia 6630, which I’ve had for more than a year. I’ve tried several aggregators, and also the mobile Bloglines, which is accessed through a web browser such as Opera, and, while they work well, they’re too slow and cumbersome for my tastes. Therefore, I used a combination of newspipe (to convert posts from feeds to email messages) and Profimail (to access a mailbox through IMAP). Recently, I’ve dumped newspipe for rss2email, for reasons I’ll mention in a future post, and that’s what I have right now.

I’ve been considering other possibilities, though. As I said, a laptop isn’t a good idea here. I’d mostly like something a little bigger than the 6630, with a larger and better screen, and possibly a QWERTY keyboard (to do annotations and so on). The Nokia E61 seems to fit the bill (and it supports Wi-Fi, which would save me a lot of money in phone bills), though I wouldn’t like to spend too much money (after all, it’s mostly a luxury – I can keep using the 6630, or even battle my distaste for reading on the PC). A PocketPC (are they still called that, these days?) PDA might also do the trick, though the ones I saw recently would cost an arm and a leg (really, 800 euros!? what are they thinking?).

So, any suggestions / tips?

If you read your email on a Series 60 phone, you probably use ProfiMail, a very nice mobile email client.

However, when you configure it to access a Gmail account, you will always get the “This site uses an untrusted certificate” message. Quite annoying, since it requires 2 key presses to pass. Every single time. There’s no option for “I know, just ignore it from now on”.

The cause of the problem is that Series 60 phones, much like web browsers, have a list of trusted root certificates, and the one Gmail uses, from Equifax, isn’t in it.

How to fix it? I search around, and didn’t find a single page with instructions on how to solve this thing. But, by combining this and this, I was able to remove that annoying prompt for good.

Here’s how to do it:

• Using the phone’s browser (Opera also works), browse to http://www.ocasta.co.uk/cert.html. Click on the only link on that page.
• It should prompt you to install the certificate. Accept it.
• It should ask you what you want to trust the certificate for. Choose “Internet”.

This works on my Nokia 6630. Other phones may simply install the certificate, after which you have to go to Settings, Security, Certificate Management, and set the new Equifax certificate as “Trusted”.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Not much longer, now…

Add the following line to /etc/postfix/main.cf:

content_filter=smtp-amavis:[127.0.0.1]:10024

and the following lines to /etc/postfix/master.cf:

# amavisd-new
smtp-amavis unix - - n - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes

127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o receive_override_options=no_header_body_checks

(re)Start postfix. It should be listening on two ports now: 25 and 10025.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Go to the amavisd-new site and download the latest version (2.3.3 at the time of writing). Uncompress it somewhere, then copy amavisd to /usr/local/sbin (for instance), and amavisd.conf to /etc.

Edit the /etc/amavisd.conf file. Add the following lines to the end:

$daemon_user = 'amavis';
$daemon_group = 'amavis';
$mydomain = 'YOURDOMAIN'; # replace with your own domain, of course
$virus_quarantine_method = '';
$spam_quarantine_method = '';
$banned_files_quarantine_method = '';
$bad_header_quarantine_method = '';
$sa_tag_level_deflt = -202.0;
$sa_tag2_level_deflt = 8.31;
$sa_kill_level_deflt = 50;
$sa_dsn_cutoff_level = 50;
$mailfrom_notify_admin = "YOUR EMAIL"; # add \ before the @, as in cats\@allyourbase.com
$mailfrom_notify_recip = "YOUR EMAIL"; # add \ before the @, as in cats\@allyourbase.com
$mailfrom_notify_spamadmin = "YOUR EMAIL";# add \ before the @, as in cats\@allyourbase.com
$final_virus_destiny = D_DISCARD;
$final_banned_destiny = D_DISCARD;
$final_spam_destiny = D_DISCARD;
$final_bad_header_destiny = D_PASS; # many badly configured servers out there
$warnvirusrecip = 1;
$warnbannedrecip = 1;

(a few lines may have wrapped around in the above. They all start with “$“)

Create the following directories, making sure they belong to user and group amavis:

/var/amavis/tmp
/var/amavis/var,
/var/amavis/db

Run amavisd, and check if it’s listening in port 10024. If not, there should be some error message telling you what the problem (in /etc/amavisd.conf) is.

We’re almost done, now. Next: configuring Postfix to work with amavisd-new.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Now for the anti-virus. Go to the ClamAV site, download the latest stable version, uncompress it, then compile and install it:

./configure --with-user=amavis --with-group=amavis --sysconfdir=/etc
make
make install

(note: 3 lines. The first one ends with “–sysconfdir=/etc”)

Edit the /etc/freshclam.conf file. At the end, add the following line:

DatabaseMirror db.XY.clamav.net

replacing the XX with your country code (e.g. “us”, “uk”, “pt”, etc.)

Now, edit the /etc/clamd.conf file.

Near the beginning, comment out or delete the “Example” line. Then, add the following lines at the end:

TCPAddr 127.0.0.1
User amavis

Test if ClamAV is able to update itself:

freshclam --log-verbose

If there are any problems, it should tell you.

Finally, make it so that

/usr/local/bin/freshclam -d

(note the “-d”)

and

/usr/local/sbin/clamd

are run when the system boots. That will depend on your Unix variant.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Thought I’d forgotten about this one, didn’t you?

Ready to make SpamAssassin actually use MySQL for the bayes database?

Start by creating the database itself:

mysql
CREATE DATABASE bayes;
GRANT ALL PRIVILEGES on bayes.* TO bayes@localhost IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES on bayes.* TO bayes@10.0.0.1 IDENTIFIED BY 'password';
EXIT

changing password to something else, of course

Next, I know you’ve already installed SpamAssassin using CPAN, but go to www.spamassassin.org and download it manually; you’ll be needing a file from the distribution, and while it should still be in /root/.CPAN, it’s simpler this way. Uncompress the .tar.gz and go to the sql/ directory. Then type:

mysql -u bayes -p < bayes_mysql.sql

It’ll ask for a password, which is the one you used when creating the database.

Now, edit the file /etc/mail/spamassassin/local.cf. Add the following lines:

bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn DBI:mysql:bayes:localhost
bayes_sql_username bayes
bayes_sql_password password

(again, replace password with the proper one.)

SpamAssassin is now configured to store bayes data on MySQL. Wasn’t too hard, was it?

Much like Firefox, the last RC – in this case, 1.5 RC 2 – is the same as the final version. If you already have that one, you have the latest Thunderbird.

For everyone else, you can get Thunderbird here.

Ricardo got there first this time. Mozilla Thunderbird 1.5 RC1 is out, and can be downloaded here. Or you can try the auto-update.

I’m using it right now, at work (on Linux).

(NOTE: this is part of the “An Anti-Spam gateway” series)

Time for MySQL. We’ll be using it just for storing the bayes tokens, not for per-user configuration (because, in this case, there is none) or anything else.

Note: if you already have a working MySQL on the server, being used for something else (this isn’t Windows, you can use the same machine for several different things ), just skip to the next part.

So, go to http://dev.mysql.com/downloads/, choose the General Availability Release (it’s the top one – at this moment, it’s version 5.0), then scroll down a lot until you find the Source Downloads. Get the .tar.gz file. Uncompress it, enter the directory, and then…

export CFLAGS="-O2"
export CXXFLAGS="-O2 -fno-exceptions -fno-rtti"
./configure --prefix=/usr/local --localstatedir=/var/mysql --sysconfdir=/etc --without-innodb

(note: your browser may have split the “./configure” line in two. It starts with “./configure”, and ends with “–without-innodb”.)

then do the usual

make
make install

If everything went well,

mysql_install_db

and start the server with:

mysqld_safe --user=mysql &

Then assign a password to the MySQL root account (which is not the OS’s root account!):

mysqladmin -u root password "newpwd"
mysqladmin -u root -h host_name password "newpwd"

Replace “host_name” with the correct one, and “newpwd” with something else, of course.

Now, for efficiency, you should have an /etc/my.cnf file with several options set manually. The MySQL sources include some examples in the support-files directory: my-small.cnf, my-medium.cnf, etc.. Copy one to /etc/my.cnf, edit it, then restart the server. Nope, I won’t give detailed instructions for that here… read the MySQL docs at http://www.mysql.com.

Next: how to create the bayes database, and how to configure SpamAssassin to use it.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Vipul’s Razor is a piece of software for accessing a large database of spam messages. By installing it, and configuring SpamAssassin to use it, you can, among SpamAssassin’s other tests, see whether a particular message has been previously reported as spam by other people. You can also report messages yourself.

It’s not a perfect system, but the best way to use SpamAssassin is by combining several methods of checking spam. This is simply one more.

Note: although I have previously said that I won’t be offering different alternatives, as this is a “recipe”, for Razor I’ll make an exception. If you don’t want to use it, simply skip this part and move on to the next.

You’re still here? Fine, let’s move on. Go to CPAN’s shell again with the command:
perl -MCPAN -e shell

then type in the following commands, to install some more Perl modules:

install Time::HiRes
install Digest::SHA1
install Test::Simple
install Test::Harness
install Getopt::Long
install File::Copy
install URI::Escape
quit

Again, it’s possible that some are already installed, and it’s also possible that some ask for dependencies; if so, say that, yes, indeed, you know what you’re doing and that you really, certainly, indeed, want to install them.

Now, go to http://razor.sourceforge.net/, and download “razor-agents“. Just that one, not the -sdk file. Uncompress it, enter the directory, then:


perl Makefile.PL
make
make install


It’s installed, now. But we have to configure something more, and, first, we need to create an user. Let’s call it “amavis“, since it will be amavisd-new which will be running the show, and, therefore, everything except Postfix itself will run as that user.

So, create a group “amavis” and an user “amavis” with that group as its main group, and /var/amavis as its home directory. Come on, you don’t need help with this one, do you?

Then su to that user.

Now, execute the following (as user “amavis):

razor-admin -register

Finally (for SpamAssassin 3.1 or greater) , go back to root and edit the file /etc/mail/spamassassin/v310.pre. Uncomment the following line

#loadplugin Mail::SpamAssassin::Plugin::Razor2

by removing the “#” from the beginning.

There. Next part: MySQL.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Installing SpamAssassin isn’t too hard, thanks to Perl‘s CPAN.

First, type the following:
perl -MCPAN -e shell

If it’s the first time you do that, it will ask you a bunch of questions. Accept the defaults, and when asked about where to download from, try to pick a couple of places near you. When you finally get to the command prompt, it’s time to install the modules.

Now, we could simply tell it to install SpamAssassin, and it would install all the required modules for it. But we’ll want to install some optional ones as well, so, do the following (one at a time, naturally), saying “yes” whenever it asks you if you want to install a pre-requisite (note, also, that some of the modules may already be installed – if so, just move on to the next one):

install MIME::Base64
install DB_File
install Net::DNS
install Net::SMTP
install DBI
install LWP
install Compress::Zlib
install IO::Zlib
install Archive::Tar
install Mail::SpamAssassin


There are 2 still missing, related to MySQL, but we haven’t installed MySQL yet, so… just type quit to exit the CPAN shell.

Onward to Razor, in the next part.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Continuing the Postfix configuration… Edit the /etc/postfix/main.cf file.

Look for, and change according to your configuration, the following options:

myhostname (should be your machine’s name)

myorigin (probably $mydomain)

mynetworks (your network(s), possibly 10.0.0.0/24 in this case)

relay_domains (should be, of course, your domain(s))

bounce_notice_recipient, delay_notice_recipient, error_notice_recipient, policy_notice_recipient (change them all to postmaster@yourdomain, assuming that address exists or is an alias to an existing one – possibly yours, and replacing yourdomain with your domain, of course)

Change the smtpd_client_restrictions option to:
  smtpd_client_restrictions = permit_mynetworks,
   reject_rbl_client relays.ordb.org,
   reject_rbl_client sbl.spamhaus.org,
   permit

Change smtpd_helo_required to yes. Mail servers should learn to behave. Besides, it’s bulk spam senders that normally don’t.

Now, save that file, and edit /etc/postfix/transport. Except for the comments, it should be changed so that it only has this line:

*       smtp:10.0.0.2

(remember? that’s the real mail server’s address)

Leave the editor, then do the following: postmap /etc/postfix/transport, to update the transport.db file.

Now, test it. Yes, when you start it (postfix start), if everything went well, it should already be working as a non-filtering gateway. Assuming, of course, that you configure your firewall so that mail from the outside (to port 25) is delivered there, and that it can connect to the real mail server, and that the latter accepts mail from the gateway.

When it’s working, go to the next part of the series.

(NOTE: this is part of the “An Anti-Spam gateway” series)

So, at last, we’re getting our hands dirty. It’s time to separate the boys from the men, or something.

Now, as I was writing this, I had a decision to make. Should this part, called “Postfix”, include all the Postfix-related instructions, which will produce a non-working, non-testable mail server until the rest of the stuff is installed? Or should this part have enough to create a standard gateway, not depending (yet) on SpamAssassin, ClamAV and the rest – and, only in THOSE parts, will the instructions for configuring Postfix to use them appear?

I’ve decided to do it the latter way. It’s important to be able to test each part of the gateway as you install it, instead of doing everything and only “turning it on” at the end, with a myriad possible points of failure.

So, to install Postfix: go to http://www.postfix.org/download.html, choose the closest mirror, and download the most recent stable version of Postfix (at this time, it’s 2.2 patchlevel 5, or 2.2.5). Uncompress it, enter its directory, and then…

AUXLIBS='-ldb' CCARGS='-DHAS_DB' OPT='-O2' DEBUG='' make
AUXLIBS='-ldb' CCARGS='-DHAS_DB' OPT='-O2' DEBUG='' make install

There, an installed Postfix. (accept the defaults at the end.) The configuration files are at /etc/postfix, so let’s go there and make a few changes to main.cf… in the next part, since this one is becoming too long.

(NOTE: this is part of the “An Anti-Spam gateway” series)

These days, most Linux distributions, especially the RPM-based ones like Fedora, Red Hat or SUSE, increasingly assume that “nobody compiles stuff anymore”. Due to that, they don’t install, by default, the development parts of most libraries and applications.

An example: by default, a distro will install OpenSSL, with the openssl package. But that’s the library files only. It “gives” OpenSSL to other RPMs that need it, but, when you try to compile any program to use OpenSSL, it will fail (or, possibly worse – it will compile, but without OpenSSL support, and you may fail to notice it), because you are missing the header files.

They’re in the openssl-devel package.

So, from now on, through the rest of this series, pay attention when compiling (mostly in the ./configure part). If it fails, or if it passes but says that you are missing an important library, the thing to do is probably to look for the missing *-devel package, and install it.

Incidentally, this is not a problem in the BSDs.

(NOTE: this is part of the “An Anti-Spam gateway” series)

Before we begin to actually get our hands dirty , here are a few things to note:

• As I said in the introduction, this is a recipe, not a reference manual. There are certainly many other ways to do something like this, but I won’t be exploring them.
• I’m not explaining basic stuff like “to uncompress a .tar.gz file, type tar xzf filename“. If you don’t know things like that, this series really isn’t for you. Sorry about that.
• The required hardware depends on the company / organization size, and on how much mail you receive. A Pentium III with 128 MB of RAM is more than enough for a company with 1000 employees. No, that’s not a typo – if you’re not using Windows or Oracle (yuck), you don’t need 64-CPU supercomputers with terabytes of RAM.
• The operating system should be a Unix-like system with a C compiler such as gcc. Most Linux distributions include everything you need, although you may have to manually select the “Development” packages during installation, or add them later. Other alternatives include OpenBSD (my personal favorite, and the server this blog runs on), FreeBSD, NetBSD, Solaris, etc.. Anything but Windows or SCO.
• I will be compiling most of the software manually. Using packages (such as RPMs, or BSD ports) is possible, but they may not activate some needed option by default. It’s up to you.
• This machine will be a gateway — that is, it won’t have mailboxes, it will just receive mail from the Internet, refuse messages from known open relays, tag spam messages by prepending “*** SPAM ***” to the message subjects, and stop emails with viruses, warning the recipient that someone tried to send them a virus. Then, if the mail wasn’t stopped, it delivers it (or the virus warning message) to the “real” email server, which has the actual mailboxes.
• For simplification, I’m assuming that the machine has only one network card, and has the IP adress 10.0.0.1 (supposedly, some firewall is redirecting port 25 from the outside to this address). The “real” mail server has the address 10.0.0.2. More complex configurations are possible, but, again, I’m not exploring them here.
• I’m also assuming that the gateway has unrestricted outbound Internet access. (it only needs inbound access to port 25). After it’s working, you may possibly want to restrict it somewhat (though I think it it’s a bit useless), but it’ll always need to do DNS queries, access some spam databases like Razor, and update ClamAV.

So, before you go to part 2, you should have a box with one network card configured as 10.0.0.1, running an Unix-like OS with a C compiler.

(NOTE: this is part of the “An Anti-Spam gateway” series)

“An Anti-Spam gateway” is the Tlog’s third series.

In it, I will write about how to use a Unix-like free operating system such as Linux or OpenBSD, using Postfix, amavisd-new, SpamAssassin with MySQL, and ClamAV to create a free, open source, efficient mail gateway, for tagging spam (I don’t believe in deleting it automatically, that should be up to the users themselves) and stopping email viruses.

While I plan to make this not too advanced, I won’t deal with the basics here, like the installation of the OS, or uncompressing files, and so on.

Also, I won’t explore all the alternatives. This is a HOWTO, not a reference manual. Think of it as a recipe: while a cook is free to change some of the ingredients or procedures, the recipe doesn’t usually say “if you do this instead, that will be the result”. For instance, you may not require an anti-virus, or may want to use a different one, or may want to delete spam messages above a certain score, or may not use MySQL for the Bayes database… all of those possibilities are perfectly fine. I just won’t explore them here.

I hope this series is useful. Few companies, except perhaps multinationals, have gateways as good as this one will be. Trust me. I may be missing out the money I could make by selling “consulting” to companies to do this, instead of just posting the information on the net; however, I don’t want to make a living of installing and configuring email gateways , and, also, if the majority of companies (and possibly even ISPs) adopt something like this, then there may be, in the future, less spam out there. Making the world a better place, and all that stuff. A guy can dream.

Mozilla Thunderbird 1.5b2 is out (saw it on Ricardo Saramago’s blog).

Note: the “what’s new” list in the 1.5b2 page lists what’s new since 1.0.x, not since 1.5b1, contrarily to what it looks like.

As I don’t use an email client at home anymore, I’ll only test this version when I get back from holidays…

Does GMail have competition?

From what I can see, it looks like a “normal” email program… but in the browser. And they say it’s responsive like an email program.

I have 2 Yahoo! Mail accounts, though I don’t use them these days – I use fetchyahoo to redirect them to my GMail account. But if I get into the beta… I may say something more here.

Shortly after Firefox 1.5b1, the new beta of Mozilla Thunderbird is also out.

I don’t use an email client at home (I use GMail), but I’ve just updated Thunderbird from 1.0.6 to 1.5b1 at work (Linux). So far, so good. No crashes (these betas have been at least as stable as the latest stable versions, which is impressive). New preferences look, just like Firefox. Spell checking as you type (may be of interest to many people, but I turned it off in less than a minute ).

Not a lot of “wow” stuff here, but then again, it’s a mail client.