48 hour anti-spam deathmatch: Thunderbird vs. Spamassassin
Where one lives on the internet, one gets spam. Back in the day (~2001) I ran all my email on my old domain through Spamassassin, and it did a fairly good job of keeping up with all the junk I managed to receive. Bayesian filters were still new and still unencumbered by confusing hashbuster attacks. I haven’t used any anti-spam but Thunderbird’s built in system for quite some time though, but earlier this week I decided it wasn’t nearly effective enough for my spam-gobbling needs. So I had Thunderbird’s eminently trainable junk filter work in conjunction with Spamassassin to weigh how they performed alongside one another. Here’s how I worked it:
- Spamassassin only quarantines spam ranked 10 (on a scale of 0-10) — insanely spamalicious. Anything it ranks 1-9 gets [SPAM] added to the subject line. Tbird doesn’t get to analyze spamalicious emails.
- Thunderbird still filters spam, but does not auto-quarantine or tag email with [SPAM] in the subject line.
- Both solutions are working inline to defend against spam. In other words, email can (but won’t necessarily) be tagged as spam twice: once by Tbird, once by SA.
- For reference: both solutions have to do their best to tell the difference between real spam and PR spam, which, in all reality, look dangerously similar sometimes.
So what are the results after 48 hours? (Please note: these numbers are not scientific!)
Email received: roughly 580, including spam.
Thunderbird
Positives: 31
False positives: 3
False negatives: 16
Spamassassin
Positives: 19
False positives: 15
False negatives: 28
Quarantined (Tbird couldn’t scan these): ~110
Conclusion: they both kind of suck, but Thunderbird seems to suck significantly less, especially in the false positives department. Although neither seems to match the spam-killing power of Akismet and Gmail, which rarely (read: maybe a few times a year) seem to get false positives or negatives. Bummer for us client-using suckers!
Final notes: I live in email, and it’s all spam. When I’m out and about checking email on my phone without even my not-so-fantastic Thunderbird spam filter, I realize that my signal to noise ratio is probably about about 8:1, and will soon be 4:1 (and eventually become 1:1). Email is badly broken, and desperately needs to be fixed. I’m sure I’m not the only one that would drastically change the way I use email if it meant everyone could start using a better, open, verifiable system to stamp out spammers.
Co-founder of 
Here here! I’ve gotten approximately 11,000 spam emails in the last month and I really feel your pain. So far my SpamBayesian filter through Outlook seems to be catching 85% of these, the rest are easily filterable, but still. Wow, 11k spams a month?
If there was a key you could send your friends to embed into their emails sent to you, sort of a key, that could be configured in the Outlook/TBird contact info and automatically maintained through messages sent and replied, and automatically filtered, this might be a step in ensuring the important things at least come easily recognizable, then it’s just a matter of sifting through the trash for the last .1% of important emails. :)
-Jeff
Until I recently applied for loans, Gmail did a great spam-filtering job. Now, since the loan apps, I get spammed with about a hundred loan emails a day. About five slip through per week. Not too shabby. And those, Mac Mail 2 picks up for me as Junk.
I have found that Codeodes Cactus Spam Filter works very well after being trained. I don’t get seem to have a problem with false positives or negatives (a few). Of course you need to be doing POP3 mail.