Slamming Spam with SpamAssassin
This chapter is from the book
This chapter is from the book
This chapter is from the book
In this chapter, we introduce SpamAssassin, which is a widely used interface to other anti-spam mechanisms as well a spam classifier in its own right. SpamAssassin has its own rules base, which is used to assign a "score" to each mail message. The methods SpamAssassin uses to classify whether email is spam are as follows:
- Header analysis —SpamAssassin can analyze these headers of an email message and generate a score based on them.
- Body analysis —The body of a spam message often contains phrases (i.e., "Reduce your debt!") that can easily identify it as spam.
- Bayesian analysis —Bayes is a statistical methodology where both the headers and body of a message are compared against a known database of phrases. This book dedicates several chapters to Bayesian analysis due to its effectiveness.
- Distributed checksums (Vipul's Razor and DCC) —In this process, a message is check-summed and put into a distributed database along with a count. The higher the count for the checksum associated with the message, the more likely the message is spam.
- Blackhole listing checking (MAPS RBL, etc.) —Blackhole lists are ranges of IP addresses where spam originates. If a message comes from such a network, the message score can be adjusted accordingly.
- Automatic whitelisting/blacklisting (AWL) —SpamAssassin can automatically add email addresses to databases of whitelists or blacklists.
- Manual whitelisting/blacklisting —The software can accept/reject known good/bad email addresses.
SpamAssassin rules can be defined by the administrator and by the user if certain conditions are met. SpamAssassin's power is evident when the individual scores are summed—the final score is worth much more than the individual scores themselves. After SpamAssassin has "scored" a message, the tool can do a number of things:
- Add headers (for example, a spam score like X-SpamAssassin-Score: 40)
- Modify existing mail headers (i.e., change the subject line)
- Send a new message regarding the spam (i.e., report the spam to someone)
- Submit the spam to a database for tracking purposes
- Send a message to the sender (spamtrap-related keywords)
Arguably the most useful function in SpamAssassin is the act of scoring the message and adding an appropriate header indicating the score. Adding a score enables email clients (such as Outlook, Mozilla Messenger, etc.) to filter mail into folders as the end user wishes. However, setting up and training end users takes considerable time and effort from the administrator.
Our coverage of SpamAssassin is designed for Unix-based systems. If you are interested in running SpamAssassin on a Microsoft Windows product such as XP, you should read the writeup available at http://www.openhandhome.com/howtosa260.html . This web page covers the largely manual process for making SpamAssassin work on MS Windows platforms.
SpamAssassin can be set up to work at mail delivery time (when mail is being written to the users' mailbox, often called "per-user") or transfer time (when mail is coming into the system). Both are useful in the fight against spam, although normally the administrator picks one approach per system. Per-user checking is good for a small number of users and for working out bugs. Figure 3.1 shows a possible SpamAssassin integration with a mail system on a per-user basis.
)
Figure 3.1 SpamAssassin deployed per user.
When you are happy with the per-user installation, you can roll the setup system-wide and implement SpamAssassin for all user accounts at mail transfer time. SpamAssassin documentation refers to this mode as "site-wide", but that is a bit of a misnomer, which is why this book will use the term interchangeably with "system-wide". Figure 3.2 illustrates SpamAssassin integration at MTA time (system-wide).
)
Figure 3.2 SpamAssassin deployed system-wide.
The box labeled MTA interface in Figure 3.2 is the mechanism that calls SpamAssassin, which is the topic of the next section.
SpamAssassin and MTA Integration
Before we get to the actual installation of SpamAssassin, it is useful to cover how to integrate it into your environment. Each MTA has its own way of adding in third-party components like SpamAssassin. In addition, each MTA itself can have several methods of integrating SpamAssassin into it.
SpamAssassin is often integrated into a large email infrastructure via an MTA filter such as milter, MIMEdefang, and Qmail-Scanner. These programs enable a number of actions on messages, including anti-virus, filtering, and, of course, anti-spam. The other way to integrate SpamAssassin into your email system is to use a facility such as Procmail, which can be enabled on a per-user basis. Using Procmail might be good for initial testing or for use on a small site. Although this approach will certainly work, the per-user procmail method requires more work on the administrator's part. Any site with more than a handful of users is probably going to want to use a site-wide method.
In the case of Sendmail, there is a standard facility called Sendmail Mail Filter (or milter), which allows administrators to call programs to perform desired functions (including filtering and spam detection), which are not part of Sendmail itself. We use milter and MIMEdefang as the method of integration between Sendmail and SpamAssassin.
With Postfix, the options are more varied. SpamAssassin integration methods could include MailScanner, MIMEdefang, or amavisd-new, among others. We chose to use amavisd-new as our Postfix integration method.
Regarding qmail, the options are fewer. We chose the Qmail-Scanner software as our integration. As with other MTAs utilizing Procmail for individual users, qmail's .qmail function could be used to call a script to invoke SpamAssassin. This would be good for a small organization, but larger sites will want to use something like Qmail-Scanner.
Table 3-1 summarizes what we use in this chapter for implementing SpamAssassin from each MTA and where to get more information.
Table 3-1.
|
MTA |
Interface(s) |
More Information |
|
Sendmail |
MIMEDefang |
|
|
milter |
||
|
Postfix |
amavisd-new |
|
|
qmail |
Qmail-Scanner |
|
|
maildrop |