SpamAssassin

SpamAssassin

by Alan Schwartz
Released July 2004
Publisher(s): O'Reilly Media, Inc.
ISBN: 9780596007072

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

The annoyance factor for individual users whose email is crammedwith pitches for pornography, absurd moneymaking schemes, anddubious health products is fierce. But for organizations, the costof spam in lost productivity and burned bandwidth is astronomical.While society is grappling with a solution to the burgeoning crisisof spam proliferation, the pressure is on system administrators tofind a solution to this massive problem in-house. And fast. Sysadmins can field scores of complaints and spend months testingsoftware suites that turn out to be too aggressive, too passive, ortoo complicated to setup only to discover that SpamAssassin (SA),the leading open source spam-fighting tool, is free, flexible,powerful, highly-regarded, and remarkably effective. The drawback?SpamAssassin's lack of published documentation. SpamAssassinby Alan Schwartz, is the only published resource devoted toSpamAssassin and how to integrate it effectively into yournetworks. This clear, concise guide clarifies the installation,configuration, and use of the SpamAssassin spam-checking system(versions 2.63 and 3.0) for Unix system administrators using thePostfix, Sendmail, Exim, or qmail mail servers, helpingadministrators make the right integration decision for theirparticular environments. It covers concrete advice on how to:

  • Customize SpamAssassin's rules, and even create new ones TrainSpamAssassin's Bayesian classifier, a statistical engine fordetecting spam, to optimize it for the sort of email that youtypically receive

  • Block specific addresses, hosts, and domains using third-partyblacklists like the one maintained by Spamcop.net.

  • Whitelist known good sources of email, so that messages fromclients, coworkers, and friends aren't inadvertently lost.

  • Configure SpamAssassin to work with newer spam-filteringmethods such as Hashcash (www.hashcash.org) and Sender PolicyFramework (SPF).

    • Sys admins, network administrators, and ISPs pay for spam withhours of experimentation and tedious junk email management, frayeduser tempers, and their sanity. SpamAssassin, together with thisessential book, give you the tools you need to take back yourorganization's inboxes. "Detailed, accurate andinformative--recommended for spam-filtering beginners and expertsalike." --Justin Mason, SpamAssassin development team

    Publisher resources

    View/Submit Errata

    Table of contents

    1. Copyright
    2. Preface
      1. Scope of This Book
      2. Versions Covered in This Book
      3. Conventions Used in This Book
      4. Using Code Examples
      5. Comments and Questions
      6. Acknowledgments
    3. 1. Introducing SpamAssassin
      1. 1.1. How SpamAssassin Works
      2. 1.2. Organization of SpamAssassin
      3. 1.3. Mailers and SpamAssassin
        1. 1.3.1. Scanning at the MTA
        2. 1.3.2. Scanning at the MDA
        3. 1.3.3. Scanning with a POP Proxy
        4. 1.3.4. Scanning at Multiple Places
      4. 1.4. The Politics of Scanning
    4. 2. SpamAssassin Basics
      1. 2.1. Prerequisites
      2. 2.2. Building SpamAssassin
        1. 2.2.1. What Gets Installed
        2. 2.2.2. Basic Configuration
        3. 2.2.3. Testing SpamAssassin
        4. 2.2.4. SpamAssassin Options
          1. 2.2.4.1. Locating configuration files
          2. 2.2.4.2. Scripting and testing options
          3. 2.2.4.3. Untagging
          4. 2.2.4.4. Reporting
      3. 2.3. Invoking SpamAssassin with procmail
      4. 2.4. Using spamc/spamd
        1. 2.4.1. Setting up spamd
          1. 2.4.1.1. Connection type
          2. 2.4.1.2. Running as a non-root user
          3. 2.4.1.3. Other security features
          4. 2.4.1.4. Locating configuration files
        2. 2.4.2. Testing spamc
        3. 2.4.3. spamc Options
          1. 2.4.3.1. Connection type
          2. 2.4.3.2. Handling problems
        4. 2.4.4. Invoking spamc with procmail
        5. 2.4.5. Changing SpamAssassin Configuration Files
      5. 2.5. Invoking SpamAssassin in a Perl Script
      6. 2.6. SpamAssassin and the End User
        1. 2.6.1. True Negatives (ham)
        2. 2.6.2. True Positives (spam)
        3. 2.6.3. False Positives
        4. 2.6.4. False Negatives
    5. 3. SpamAssassin Rules
      1. 3.1. The Anatomy of a Test
      2. 3.2. Modifying the Score of a Test
        1. 3.2.1. Modifying Scores Systemwide
        2. 3.2.2. Modifying Scores on a Per-User Basis
        3. 3.2.3. Storing Scores in an SQL Database
        4. 3.2.4. Storing Scores in an LDAP Database
      3. 3.3. Writing Your Own Tests
        1. 3.3.1. Header Tests
          1. 3.3.1.1. Configurable header tests (SpamAssassin 3.0)
          2. 3.3.1.2. check_rbl( )
          3. 3.3.1.3. check_rbl_txt( )
          4. 3.3.1.4. check_rbl_sub( )
        2. 3.3.2. Body Tests
        3. 3.3.3. URI Tests
        4. 3.3.4. Meta Tests
      4. 3.4. The Built-in Tests
        1. 3.4.1. 10_misc.cf
        2. 3.4.2. 20_fake_helo_tests.cf
        3. 3.4.3. 20_body_tests.cf
      5. 3.5. Whitelists and Blacklists
        1. 3.5.1. Systemwide Whitelists
          1. 3.5.1.1. Whitelisting senders
          2. 3.5.1.2. Whitelisting senders by relay
          3. 3.5.1.3. Whitelisting recipients
        2. 3.5.2. Systemwide Blacklists
        3. 3.5.3. Per-User Whitelists and Blacklists
    6. 4. SpamAssassin as a Learning System
      1. 4.1. Autowhitelisting
        1. 4.1.1. Principles
        2. 4.1.2. Configuration
          1. 4.1.2.1. Configuring per-user autowhitelists
          2. 4.1.2.2. Configuring a system-wide autowhitelist
        3. 4.1.3. Using an Autowhitelist
      2. 4.2. Bayesian Filtering
        1. 4.2.1. Principles
        2. 4.2.2. Configuration
        3. 4.2.3. Training
        4. 4.2.4. Daily Use
          1. 4.2.4.1. Ongoing training
          2. 4.2.4.2. Expiration and importing
        5. 4.2.5. Storing Bayesian Data in SQL
        6. 4.2.6. A Sitewide Bayesian Classifier
    7. 5. Integrating SpamAssassin with sendmail
      1. 5.1. Spam-Checking at Delivery
      2. 5.2. Spam-Checking During SMTP
        1. 5.2.1. The Milter Interface
        2. 5.2.2. MIMEDefang
          1. 5.2.2.1. Installing MIMEDefang
          2. 5.2.2.2. Starting the MIMEDefang multiplexor
          3. 5.2.2.3. Checking multiplexor status
          4. 5.2.2.4. Starting the MIMEDefang milter
          5. 5.2.2.5. Verifying the MIMEDefang processes
          6. 5.2.2.6. Customizing MIMEDefang
        3. 5.2.3. SpamAssassin Integration
          1. 5.2.3.1. Adding sitewide Bayesian filtering
          2. 5.2.3.2. Adding sitewide autowhitelisting
          3. 5.2.3.3. Adding per-domain or per-user streaming
      3. 5.3. Building a Spam-Checking Gateway
        1. 5.3.1. sendmail Configuration
        2. 5.3.2. SpamAssassin Configuration
        3. 5.3.3. MIMEDefang Configuration
        4. 5.3.4. Routing Email
        5. 5.3.5. Internal Server Configuration
        6. 5.3.6. Testing
    8. 6. Integrating SpamAssassinwith Postfix
      1. 6.1. Postfix Architecture
      2. 6.2. Spam-Checking During Local Delivery
      3. 6.3. Spam-Checking All Incoming Mail
        1. 6.3.1. Using a Program as a Content Filter
        2. 6.3.2. Using a Daemon as a Content Filter
        3. 6.3.3. Filtering Before Address-Rewriting
      4. 6.4. Building a Spam-Checking Gateway
        1. 6.4.1. Installing amavisd-new
        2. 6.4.2. Configuring amavisd-new
          1. 6.4.2.1. Essential options
          2. 6.4.2.2. MTA options
          3. 6.4.2.3. Daemon process options
          4. 6.4.2.4. Distinguishing local domains
          5. 6.4.2.5. Postfix-specific options
          6. 6.4.2.6. Logging options
          7. 6.4.2.7. Spam-handling options
          8. 6.4.2.8. Recipient whitelists
          9. 6.4.2.9. Sender whitelists and blacklists
          10. 6.4.2.10. SpamAssassin settings
          11. 6.4.2.11. Storing recipient preferences in external databases
        3. 6.4.3. Basic Operations
        4. 6.4.4. Adding Sitewide Bayesian Filtering
        5. 6.4.5. Adding Sitewide Autowhitelisting
        6. 6.4.6. Routing Email Through the Gateway
          1. 6.4.6.1. Postfix changes
          2. 6.4.6.2. Routing changes
          3. 6.4.6.3. Internal server configuration
    9. 7. Integrating SpamAssassin with qmail
      1. 7.1. qmail Architecture
      2. 7.2. Spam-Checking During Local Delivery
      3. 7.3. Spam-Checking All Incoming Mail
      4. 7.4. Building a Spam-Checking Gateway
        1. 7.4.1. Installation
        2. 7.4.2. Initialization
        3. 7.4.3. Basic Operations
        4. 7.4.4. Per-User Spam Preferences
        5. 7.4.5. Sitewide Bayesian Filtering
        6. 7.4.6. Sitewide Autowhitelisting
        7. 7.4.7. Routing Email Through the Gateway
          1. 7.4.7.1. qmail changes
          2. 7.4.7.2. Routing changes
          3. 7.4.7.3. Internal server configuration
    10. 8. Integrating SpamAssassin with Exim
      1. 8.1. Spam-Checking via procmail
      2. 8.2. Spam-Checking All Incoming Mail
      3. 8.3. Using Routers and Transports
        1. 8.3.1. Configuring the Transport
        2. 8.3.2. Configuring the Router
        3. 8.3.3. Using Per-User Spam-Checking Preferences
      4. 8.4. Using exiscan
        1. 8.4.1. Installing exiscan
        2. 8.4.2. Writing acl_smtp_data
        3. 8.4.3. Using Per-User Preferences
      5. 8.5. Using sa-exim
        1. 8.5.1. Buiding sa-exim for Static Integration
        2. 8.5.2. Building sa-exim for Dynamic Integration
        3. 8.5.3. Configuring SpamAssassin for sa-exim
        4. 8.5.4. Configuring sa-exim
          1. 8.5.4.1. Choosing messages on which to run SpamAssassin
          2. 8.5.4.2. Choosing messages on which to take antispam actions
          3. 8.5.4.3. Limiting how much of the message is fed to SpamAssassin
          4. 8.5.4.4. Allowing SpamAssassin to rewrite message bodies
          5. 8.5.4.5. Archiving messages when actions are taken
          6. 8.5.4.6. Passing SMTP senders and recipients to SpamAssassin
          7. 8.5.4.7. Setting a timeout on spamc
          8. 8.5.4.8. Handling messages that cause sa-exim errors
          9. 8.5.4.9. Teergrubing
          10. 8.5.4.10. Accepting and discarding spam
          11. 8.5.4.11. Rejecting spam
          12. 8.5.4.12. Temporarily failing spam
          13. 8.5.4.13. Archiving accepted spam
          14. 8.5.4.14. Archiving non-spam messages
          15. 8.5.4.15. Debugging sa-exim
        5. 8.5.5. Using Per-User Preferences
      6. 8.6. Building a Spam-Checking Gateway
        1. 8.6.1. Routing Email Through the Gateway
          1. 8.6.1.1. Exim domain lists
          2. 8.6.1.2. Routing changes
          3. 8.6.1.3. Internal server configuration
        2. 8.6.2. Adding Sitewide Bayesian Filtering
        3. 8.6.3. Adding Sitewide Autowhitelisting
    11. 9. Using SpamAssassin as a Proxy
      1. 9.1. Using Pop3proxy
        1. 9.1.1. Installing Pop3proxy
        2. 9.1.2. Starting Pop3proxy
        3. 9.1.3. Configuring the POP Client
      2. 9.2. Using SAproxy Pro
        1. 9.2.1. Installing SAproxy Pro
        2. 9.2.2. Starting SAproxy Pro
        3. 9.2.3. Configuring the POP Client
        4. 9.2.4. Configuring SAproxy Pro
    12. A. Resources
      1. A.1. General Spam Resources
      2. A.2. Spam-Filtering
      3. A.3. SpamAssassin
      4. A.4. Mail Transport Agents
        1. A.4.1. Sendmail
        2. A.4.2. Postfix
        3. A.4.3. qmail
        4. A.4.4. Exim
      5. A.5. Related Mail Tools
        1. A.5.1. procmail
        2. A.5.2. MIMEDefang
        3. A.5.3. amavisd-new
        4. A.5.4. sa-exim
        5. A.5.5. exiscan-acl
        6. A.5.6. qmail-scanner
    13. About the Author
    14. Colophon
    15. Copyright

    Product information

    • Title: SpamAssassin
    • Author(s): Alan Schwartz
    • Release date: July 2004
    • Publisher(s): O'Reilly Media, Inc.
    • ISBN: 9780596007072