War to SPAM with SpamAssassin's plugins


Home Page | Comments | Articles | Faq | Documents | Search | Archive | Tales from the Machine Room | Contribute | Set language to:en it | Login/Register


If you do, like me, maintainance and system administration for a company, you know that the war against spam is an ongoing (and mostly uphill) battle. This week I happend to get a huge amount of spam-mail that slipped through the filters. All the mails were like:


Some random sequence of words

< html block >
<a href="url.random">click here</a>

A quick analisys of the thing told me that:

  1. The initial random sequence was too randomic and too short to be caught by the bayesian filter.
  2. It has no sense to put those sentences in the normal SpamAssassin rules since they are always different.
  3. The domain in the url is also random.

After an healty pause to stuff a dozen or so big pins in my voodoo doll, I checked out and realized one important thing: most of these e-mails had random URLs, but the IP associated to those URLs was almost always the same.

Great, we begin with something. Now, how can I check the IP of an Url?

After a while I decided that, well, yes, I can check IP adresses using an RBL, but checking URLs is not so easy.

So I donned my Baboon-Coder's hat, and began to work to a SpamAssassin's Plugin to do exactly this.

SpamAssassin can be 'extended' using "plugins". A plugin is an object that derive from the base Plugin class and inherit a number of functions from the base object.

To make a new plugin, first of all we need to import the base object: Mail::SpamAssassin::Plugin'.

This gives us a number of basic functionalities that allow us to access all the parts of an e-mail.

So, we begin with:


#!/usr/bin/perl

package <nomedelpackage>

use Mail::SpamAssassin::Plugin;
use strict;
use bytes;

use vars qw(@ISA);
@ISA = qw(Mail::SpamAssassin::Plugin);

# constructor: register the eval rule
sub new {
	my $class = shift;
        my $mailsaobject = shift;

	# some boilerplate...
	$class = ref($class) || $class;
	my $self = $class->SUPER::new($mailsaobject);
	bless ($self, $class);

	$self->register_eval_rule("<ruletoregister>");

	return $self;
}

sub <ruletoregister> {
	...
}

This is the bare essential. Of course this doesn't do anything.

Since what we want to do is to ask a DNS to get the IP of an URL, what we need is a function that ask a DNS.

Perl has Net::DNS that does exactly that. Well, actually it does a lot more... What I need is to add use Net::DNS; at the beginning of the code and then call the right methods.


# now convert the URI into an IP
my $res   = Net::DNS::Resolver->new;

$res->udp_timeout(5);
$res->tcp_timeout(5);

my $query = $res->search($uri,'A');

if ($query) {
	foreach my $rr ($query->answer) {
		# this is not strictly necessary
		next unless $rr->type eq "A";
		return $rr->address;
	}
} else {
	return;
}

After the instantiation of a 'resolver', I set the timeout to 5 seconds. This 'cause I can't wait for the default timeout (2 minutes) while processing mails and the url is either resolved now or never.

I search all the A record for an URL. Note that I stop after the first one, since a URL with more than one A record is most probably going to be a legitimate one.

Now is a matter to add some code to get rid of the junk like protocol, ports...

Now we have a basic way to get the IP from the URL, now we need to compare them against... against what?

We need to read a list of IPs from the configuration of SpamAssassin so we have something to compare against. To do so, we need the parse_config function, inherited from the basic Plugin and automatically called.

The function read the configuration file and load a series of 'rules'.


sub parse_config {
	my ($self, $opts) = @_;
	my $key = $opts->{key};

	if ($key eq 'uriip') {
		if ($opts->{value} =~ /^(\S+)\s+(\S+)\s*$/) {
			my $rulename = $1;
			my $ip = $2;
			dbg("debug: URIIP: registering $rulename");
			$opts->{conf}->{uriip}->{$rulename} = $ip;
			$self->inhibit_further_callbacks();
			return 1;
		}
	}
	return 0;
}

The code search for rules named 'uriip' and expect a rule 'name' as first parameter and an IP as second.

Now to the checking. I use the parsed_metadata function, again, this is inherited by Plugin and automatically called.


sub parsed_metadata {

	my ($self, $opts) = @_;
	my $scanner = $opts->{permsgstatus};

	# build a list of IPs converting the URIs
	my $reg;
	my %iplist = ();
	foreach my $uri ($scanner->get_uri_list()) {
		my $ip = my_uri_to_ip($uri);
		if ($ip) {
			dbg("debug: URIIP $ip for $uri");
			$iplist{$ip} = 1;
		}
	}
	# Now check if any match any defined rules.
	foreach my $rule (keys(%{$scanner->{conf}->{uriip}})) {
		my $ip = lc($scanner->{conf}->{uriip}->{$rule});
		if($iplist{$ip}) {
			dbg ("debug: URIIP hit rule: $ip");
			$scanner->got_hit($rule, "");
		}
	}

	return 1;
}

The function receive a pointer to the 'scanner' object that holds a number of interesting informations. Throught his functions we can access a list of metadata that have been already parsed and processed, like with the get_uri_list function that allow us to get a list of URI.

Now is just the matter of passing all the Uris to our IP function to get the IP. If one of the IP matches with one we are expecting, we register an hit with the got_hit function.

The configuration file for this Plugin looks like this:


uriip IP01      94.229.65.176
body  IP01      eval:check_uriip('IP01')
score IP01      8.0

uriip IP02      209.63.57.10
body  IP02      eval:check_uriip('IP02')
score IP02      8.0

Obviously 'IP01' and 'IP02' are made-up names.

The source code contains an example config file. To work, he needs Net::DNS, so if you don't have it, you'll need to CPANit.

You'll find the source in the archive.

To use it, just copy the plugin with the others, usually in ../lib/perl/<perl-version>/Mail/SpamAssassin/Plugin/, then add the configuration file in /etc/mail/spamassassin (or wherever are your other configuration file), then add a
loadplugin Mail::SpamAssassin::Plugin::URIIP
to your local configuration file (init.pre or local.cf) and include your own config file in the list of config file (usually with an 'include' line in local.cf).

If you use spamd, you'll need to restart it.


Comments are added when and more important if I have the time to review them and after removing Spam, Crap, Phishing and the like. So don't hold your breath. And if your comment doesn't appear, is probably becuase it wasn't worth it.

No messages  this document does not accept new posts

Previous Next

Davide Bianchi, works as Unix/Linux administrator for an hosting provider in The Netherlands.

Do you want to contribute? read how.  
 


This site is made by me with blood, sweat and gunpowder, if you want to republish or redistribute any part of it, please drop me (or the author of the article if is not me) a mail.


This site was composed with VIM, now is composed with VIM and the (in)famous CMS FdT.

This site isn't optimized for vision with any specific browser, nor it requires special fonts or resolution.
You're free to see it as you wish.

Web Interoperability Pleadge Support This Project Powered By Gigan