Spam control with qmail, vmailmgr, offlineimap, and spamassassin

During my vacation for the holidays, I finally took some time to deal with my personal spam situation. I get a lot of spam. I mean a lot. That’s what happens when you have multiple (dozens) of email addresses, some of which are pushing 10 years of age. And 10 years ago, back when spam wasn’t a problem, I wasn’t hesitant to give out my email address online. Which means that nowadays there probably isn’t a single spammer’s database that doesn’t have at least one of my addresses in it. At least, that’s how it felt this last week as I received over 300 spam messages per day, every day. And it’s been at this rate for a few months now.

Whenever I whinge about my spam problem, people inevitably ask why I don’t do something about it. Most email services, large and small, provide some sort of spam filtering. The thought is: since everybody else is doing it, why don’t I?

The problem has been, for some time now, that I have a fairly esoteric email system that simply doesn’t lend itself well to spam filtering. Here’s why:

1. It’s all UNIX. The UNIX philosophy is that each tool should do only one thing, and do it well. In the world of email, this means that there is a clear separation of tasks to be done by different tools:
   • MTA: Mail Transfer Agent. Sends messages from one host to another.
   • MDA: Mail Delivery Agent. Places messages in a user’s mailbox.
   • MUA: Mail User Agent. Used by a person to read and write email.
   There are varying opinions on which of the above tools should handle spam filtering. Suffice to say that on my mail server, which supports dozens of domains for various clients with different needs, I’m not going to do filtering in the MTA. The practical issue is one of control: if filtering happens in the MTA then it’s controlled by the sys-admin, whereas if filtering happens in the MUA then it’s controlled by end users (the technical issues are much more subtle). Which brings us to…
2. I need flexibility. Sometimes I’m reading and writing email while traveling with my laptop, but without an internet connection (say, on a plane). In this circumstance I need to be able to access all of my email locally, so I need a copy on my laptop. However sometimes I’m reading and writing email on someone else’s computer, so I need to have copies of all my email stored on the server. To support this, I use a program called OfflineIMAP, which (surprisingly enough) uses the offline mode of the IMAP protocol to synchronize mail boxes. What this means is that I can read, write, delete, even edit mail on my laptop, and everything I do will end up on the server. Likewise I can read/write/delete email on my mail server and everything I do will be reflected on my laptop. The point is this: I use multiple MUAs: mutt, SquirrelMail, Symbian Series 60 Messaging, Apple Mail, Microsoft Outlook, Mozilla Mail, and so on. Which means that filtering in the MUA won’t work for me. This leaves only the MDA as a filtering option, but…
3. I run a tight ship. My MTA is qmail, since other mail servers can’t touch it security-wise. To avoid giving UNIX user accounts to every user of my email server, though, I use VMailMgr as an MDA. VMailMgr plays nicely with all the components of my system (qmail, squirrelmail, courier-imap, etc) but unfortunately it’s not very flexible. As far as I can tell, there’s no built-in method for controlling it’s delivery destination. My goal was to plug spamassassin into the MDA, and have spam messages be delivered to a “Spam” folder while non-spam is passed through to the inbox. But VMailMgr didn’t support plugging any such thing in to the delivery cycle.

So what’s a geek to do? Well, since all the components I use are open source (and I wouldn’t have it any other way), I modified them to do what I needed. First, I changed the dot-qmail file for my domain to pipe messages to a shell script (rather than directly to vdeliver). The shell script passes the message to spamassassin’s spamc, then checks the result to see if the message is spam. If so, the script sets the FOLDER environment variable to “Spam”. Then vdeliver is invoked. I enhanced vdeliver to check for a FOLDER environment variable and, if it exists, to deliver the message to the specified folder rather than the inbox. To make sure that nothing is ever lost, I use safecat to get the message out of spamc and into vdeliver.

If any of these things (my custom vdeliver, the dot-qmail script, or my deliver-method-checker) sounds useful to you, email me and I’ll send you the code.

Once I got this all hooked up things were working well, except for one thing. I have some email addresses that are just aliases for others, so they get forwarded (e.g. mail to brandt-ruby@kurowski.net is sent to brandt@kurowski.net). My original setup sent the message through spamassassin twice (once during delivery to brandt-ruby@kurowski.net and once for brandt@kurowski.net). Since spamassassin modifies the message, many things that got caught as spam during the first pass were no longer recognized as spam during the second pass. So my accuracy was only about 40%. I added a command to VMailMgr to check whether a delivery address has a real mailbox (verses being just forwarded on), and I modified my dot-qmail’d shell script to only invoke spamassassin if delivering to a real mailbox.

Now about 70% of my spam is being trapped and redirected to my “Spam” folder, which I clean out once a day (after checking for false positives, of which I’ve had zero so far). It’s far less than ideal, but I am happy to only have to manually sort through 100 spam messages in my inbox instead of 300 each day.

It looks like spamassassin is fairly customizable. Perhaps during my next vacation I’ll take a whack at improving it’s accuracy.