MadBlog
Monday 30 July 2007

DM, it's not about being good or perfect…

Raphaël, you don't carve into stone (aka pass through a GR) a proposal that has not been tested yet.

The sponsoring example you took before is excellent: it was implemented and tested without a GR. Because that's what we do with excellent ideas: we test them, and if they are proven good practice, it becomes the de facto standard. You don't force things like that.

Thursday 19 July 2007

jpg/gif/pdf Spam, what can you do ?

In answer to zobel's post, here is how to fight efficiently against those nasty spams.

Well, there is a wonderful tool, called clamav that you know already for sure. What is less known is that there are people that have had the idea to use clamav to fight spams as well. They provide constantly renewed spam signatures that fight against the jpgs/gifs/... that are too many those days.

I use this script twice a day to update my signatures, and it works well.

I use this setup on a medium sized mail server with excellent results, here are the numbers for the last 30 days. The mail server had:

 2.357.038 connections attempts
 1.841.425 mails have been greylisted[1]
 ---
   510.193 mails have been rejected
   238.869 of those thanks to clamav (~50%) 
 ---
   502.580 mails have been accepted for delivery
 1.564.130 mails have been delivered to users

As you can see, on 4 mails that are considered for delivery (after the greylisting), 1 is rejected thanks to clamav. That's 25% of the incoming mails that get simply dropped, and that has almost 0 false positives[2].

Another note about greylisting: a quick reader could think that 1.8M - (2.3M - 0.5M - 0.5M) ~= 0.5M of mails are greylisted for nothing. That's not the case at all, the 2.3M are connection attempts. And we have some SMTPs that we talk to a lot (as it's the mail server of the Alumni of my school, we talk to the school MX a lot e.g.) and some of the connections carry up to dozens of mail on a regular basis. Our estimation is that in a regular day, greylisted mails that are submitted again are around the thousands, meaning some dozens of thousands a month, which is ridiculously small. And among them, sadly, most are still spams. These good ratios exists because we use conditional greylisting: we greylist IPs that look suspcicious only. But I already talked about that, and it's not really the matter of this post.

Notes

[1] using conditional greylisting: only greylist mails that come from IPs that are listed on RBLs

[2] some companies using gif's in their employees signatures can trigger false positives, but it's fairly uncommon