== Greylisting ==

[http://projects.puremagic.com/greylisting/ Greylisting] is a technique for spam avoidance which takes place at the SMTP level of your mail server. Courier has no builtin support for greylisting, but there exists a filter API for different filter purposes (courierfilter). [http://phantom.dragonsdawn.net/~gordon/courier-patches/courier-pythonfilter courier-pythonfilter] by Gordon Messner is a python framework for developping courierfilter programs in python.

Based on the pythonfilter script [http://www.freenux.org/~mm/wordpress/?p=6 greylist.py] from Mik I have written a greylisting implementation with the ideas in  Evan Harris [http://projects.puremagic.com/greylisting/whitepaper.html whitepaper].

The improvements/changes to greylist.py in detail:

 * There are now whitelists (python anydbm dbs) for domain names, ip addresses and sender/receiver addresses. The sender/receiver whitelist consists of   hashed (MD5) email addresses. With the script {{{pythonfilter_importmailaddresses.py}}} you can import e.g. the mail addresses of your customer database to have them auto-whitelisted.

 * Envelope sender and receiver addresses are now converted to lowercase before processing to handle different notations of mail addresses.

 * To calculate the triplet (IP,sender,receiver) hash value, the class C net of the sender IP is used instead of the hole IP. This measures takes mail server pools into account where subsequent delivery attempts may orgin from different IP addresses. (Only IPv4 addresses are handled this way.)

 * Store two timestamps per greylist entry instead of one: the time when the entry was created ({{{firstTimestamp}}}) and the time when the last successfull   delivery has taken place ({{{lastTimestamp}}}). So which each successfull delivery {{{lastTimestamp}}} is updated.

 * There are also two different TTL-values for the entries: {{{_sendersPassedTTL}}} and {{{_sendersNotPassedTTL}}}. Entries whose {{{lastTimestamp}}} is older than {{{_sendersPassedTTL}}} (default 36 days) are deleted out of the list. Entries which are not "authenticated" (no second delivery attempt has taken place) are deleted after {{{_sendersNotPassedTTL}}} (default 24h). The reason is to keep list small and not fill up the database with spam entries.

 * If the used db module provides the {{{sync}}} method use it every {{{_sendersPurgeInterval}}} (default now 2 hours) to have the hash db {{{_senders}}} saved to disk regulary. I experienced problems with longer purge intervals (deleted entries were still in the database after sync), thats why I choosed 2 hours instead of 12.


You can download all the necessary files in attachment:pythonfilter-greylist.tar.bz2 . 

Inside there are 4 files:

 * greylist.py : The filter file for courier-pythonfilter, based on Maks greylist.py and Gordons comeagain.py
 * pythonfilter_importmailaddresses.py : Import mail addresses which should be whitelisted to the db {{{/var/state/pythonfilter/greylist_whitelistMailAddresses}}}
 * anydbm_dump.py : Dumps out the contents of a python anydbm db hash file (for debugging purposes)
 * anydbm_import.py : Import tab seperated ascii files (key <tab> value) into python anydbm db hash files. You can use this to import the whitelists into {{{/var/state/pythonfilter/greylist_whitelistIPAddresses}}} or {{{/var/state/pythonfilter/greylist_whitelistDomains}}}.

=== Quick start guide ===

 1. copy greylist.py to a place where the other filters the pythonfilter framework are (usually something like {{{/usr/lib/python2.3/site-packages/pythonfilter/}}}).
 2. If it doesn't exist yet, create pythonfilter directory for persistent data {{{/var/state/pythonfilter}}}
 3. Optionally alter the configuration variables (TTLs/lifetimes) in greylist.py (Normally no need to do this).
 3. Optionally whitelist email addresses
    1. Create a file {{{myAddresses.txt}}} with the email addresses you want to whitelist. One entry (mail address) per line.
    2. Call {{{$ pythonfilter_importmailaddresses.py /var/state/pythonfilter/greylist_whitelistMailAddresses myAddresses.txt}}} .
    3. Note that mail addresses are stored with use of MD5 hashes, you can't restore them out of {{{greylist_whitelistMailAddresses}}}
 4. Optionally whitelist ip addresses
    1. IP addresses are stored in cleartext, so you can use the generic {{{anydbm_import.py}}} or {{{anydbm_update.py}}} scripts
    2. E.g. issue {{{$ anydbm_import.py /var/state/pythonfilter/greylist_whitelistIPAddresses myIpAddressesToSkip.txt }}}
 5. Optionally whitelist domain names
    1. Domain names are stored in cleartext, so you can use the generic {{{anydbm_import.py}}} or {{{anydbm_update.py}}} scripts
    2. E.g. issue {{{$ anydbm_update.py  /var/state/pythonfilter/greylist_whitelistDomains additionaldomainname.com}}}
 6. Make sure that the files under /var/state/pythonfilter/ have the right owner and permission.
 7. To activate you pythonfilter filter include it in your /etc/pythonfilter.conf
 8. Restart pythonfilter framework in courier with {{{filterctl stop pythonfilter ; filterctl start pythonfilter}}}. The greylist database {{{/var/state/pythonfilter/greylisted}}} is automatically created and maintained.