LucidServerMoreMailIntegration

Summary

Currently setting up spam and anti-virus filtering takes a lot of editing of configuration files and is not easy for a new Ubuntu Server users to get set up and working. In this spec we will create a new package that will provide an out of the box working setup that provides a safe, functional mail filtering system. The design principle is that the default configuration will be conservative and prefer incomplete filtering over false positives.

This spec has several pieces:

  • Create a new package that depends on postfix, amavisd-new, clamav, and spamassassin. This package will also drop in a new amavisd-configuration file to enable spam and antivirus filtering. It will use postconf and postfix-filter-add to configure postfix to filter mail with amavisd-new
  • Add greylisting via postgrey (MIR required) and integrate it via postfix-policy-add
  • Convert the mail-server task in tasksel into a task category and have three subtasks: basic (current mail-server), mail-delivery (dovecot-postfix), mail-filter (the new package from this spec).
  • Modify dovecot-postfix to provide a standard spam folder and further modify the amavisd-new configuration to tag and deliver spam to this folder
  • Add a new postfix-rbl-add script to the postfix package
  • Debconf and tasksel integration
  • Update documentation

Release Note

Easy Mail Filtering: A new feature in Ubuntu Server Edition 10.04 (Lucid Lynx) is easy setup of spam and anti-virus filtering with Postfix, Amavisd-new, Spamassassin, Clamav, and Postgrey. By installing a single package (maybe also at install time) you can have an integrated working setup that is safe to use (by design the false-positive risk is low, but non-zero).

Rationale

Currently Ubuntu Server offers a good capability for mail filtering, but it requires editing multiple configuration files. This is documented, but error prone and intimidating for new server admins. We want to make this easy and reliable in line with Ubuntu 's core values.

User stories

Bob has an Exchange server that is creaking under the load of running spam filtering and anti-virus checks in addition to the load generated just by Exchange. He's heard a lot of Linux and wants to see if it can help. He considers putting a Linux border MTA to handle spam and anti-virus checks. He installs Ubuntu Server and selects the mail filter subtask in the installer and then (after considering slitting his wrists while configuring LDAP to integrate with the user list in Exchange - Note: consider a future spec for this in the future) he is amazed how everything is integrated and working with spam and virus checks.

Scott often hangs out in #ubuntu-server and answers questions about how to do mail server related tasks. He's pleased to find that instead of pointing people at documentation and answering questions that are almost all repetition from previous questions he's answered, now he can just tell them to install amavisd-postfix and both he and the user are happy.

Elana has been tasked with establishing the mail infrastructure for a small business. They've previously used their Internet Service Provider's (ISP) mail services, but are tired of too much spam coming in and their mail getting tossed into recipient spam folders due to the poor reputation. The business already uses Ubuntu on the desktops and so Elana chooses Ubuntu Server to minimize support complexity. She reviews the Ubuntu Server Guide and discovers she can have a fully functional single tier mail stack by installing both dovecot-postfix and amavis-postfix on a single server.

Assumptions

None of note.

Design

  • New package that modifies the configuration of postfix via postconf and postfix-filter/policy-add (existing - add new postfix-rbl-add script)
  • Package modifies amavisd-new configuration by dropping a new config file in /etc/amavis/conf.d
  • Update dovecot-postfix to provide a spam folder (design is to either reject during the SMTP session or tag and deliver (virus mail is an exception and will be dropped)).
  • Postgrey configuration still needs investigation.
  • Look at adding SPF and DKIM policy components.
  • The RBL component of the spec will be very careful to avoid recommending using any specific RBLs (although it's reasonable to expect that the examples will be well chosen).
  • The system will need to use debconf to ask some additional questions for full configuration.

This is intended to provide a gateway MTA in a two tier mail architecture or in combination with dovecot-postfix provide a complete one tier mail system.

Implementation

amavis-postfix will be a metapackage with configuration of amavisd-new and postfix in it's postinst.

postfix-rbl-add will be another script similar to postfix-filter/policy-add. There is a separate effort to develop a more general solution to postfix master.cf configuration. If this matures in the Lucid cycle in time, it will be used instead, but this effort can be completed without it.

Debconf will be used to gather additional information to get to a fully functional system.

Migration

None, for new installs only

Documentation

Update Ubuntu Server Guide to describe the new simplified process, but don't remove the details on how to set up manually (this should just work, but also shouldn't be magical). Need to review existing Postfix setup docs in the community section of help.ubuntu.com as many are out of date.

Test/Demo Plan

tbd.

Unresolved issues

There were quite a number of good suggestions for this spec. Not all of them can be implemented for Lucid (a best estimate of what can be done is described). Left over ideas will be moved to a new spec for Lucid +1. Tasksel integration is desirable, but unlikely to be reachable in the Lucid cycle.

BoF agenda and discussion

UDS discussion notes

== Upstream ==
Postfix recommends amavisd-new as a filtering option and then stack other tools with it.

== Goals ==
Provide API to /etc/postfix/master.cf
Amavisd-new already covered
Separate mail stacks for use in different environments (someone wants mail filtering, someone doesn't, same goes for SASL, etc...)

== Stack ==
* MTA
* Antispam
  * RBL
  * RulesDuJour
  * Bayesian
  * Greylisting (postgrey in Main)
  * SPF/DKIM
  * Other possible rules:
    * reject_invalid_hostname 
    * reject_non_fqdn_hostname
    * reject_non_fqdn_sender
    * reject_unknown_sender_domain
    * etc
* AntiVirus
    * ClamAV
* Spam/Ham training by the user
* What do to with:
  * Spam mails
  * Virus mails
   * Special case for .exe files (very annoying for users)
* Mail delivery
  * Sieve is a system for server-side, user-defined mail filtering on header or content. It is defined by RFC 5228, and there is an implementation of the protocol in Dovecot. Sieve is thus available in the current Ubuntu mail stack, but needs to be configured by the mail user.
  * ManageSieve is a server that allows certain mail clients to create Sieve scripts on the server. It is not currently configured, but there are several implementations in Ubuntu's archive.
== Decision tree ==
Decision for mail task

 Single server?
  Y: -> everything is installed
  N: Filter or MTA ?
    Filter: Postfix + Amavisd + ...
    MTA: Postfix + Dovecot + ...
  
== Discoverabilty ==
 --> talk about it in the server-tips session and ask Ivanka :D 
 --> add url for additional info in each debconf question
 --> Tasksel option(s)
 
== Documentation ==
 * Update Server Guide with new additions, but leave old instructions.
  * Advertise the Server Guide in the conf files
  * Explain the general policy: why it's this way by default, and what steps to do to improve filtering
 * Update help.u.c mail sections for current postfix, dovecot, etc.

CategorySpec

LucidServerMoreMailIntegration (last edited 2009-12-11 06:06:19 by static-72-81-252-22)