AntispamLab is a tool for automated testing (and evaluation) of email spam filters, developed in LCA lab at EPFL. It is preferably used on PlanetLab. Keywords: antispam lab, email, spam, filter, tool, testing, evaluation.
Use of the tool is completely automated:
More realistic test as compared to related work:
To learn more about the AntispamLab tool, see our paper [PDF] published at CEAS 2007 (The Fourth Conference on Email and Antispam, Mountain View, California, USA, August 2007).
Usage requirements Downloads Usage instructions Usage examples More info Project team Acknowledgements Contact information
Only per-email-server filters are currently supported, whereas per-email-client filters testing would require additional tool development.
To use the tool, you should have a PlanetLab account. An alternative is to create a set of Fedora Core 4 virtual machines locally, for which you need one or more real Linux machines on a local LAN with installed Xen. Another but less realistic alternative is to use many real Fedora Core 4 machines on a local LAN or even distributed over the Internet. For more details, see the README.txt file from the downloads section.
IMPORTANT NOTES:
- The source code of the tool is published under the GPL terms but for non-commercial use only. GPL stands for GNU General Public License. The "non-commercial use" constraint doesn't prevent from using our tool to test commercial spam filters (you are welcome to do so!); it only prevents from building and selling commercial tools or services for testing spam filters, which are based on our free tool.
- This software is not maintained up to date with respect to the changes in AntispamLab. Updates of the version of the operating system used for PlanetLab virtual machines may require updates of the list of modules that AntispamLab automatically imports and installs onto the used virtual machines (the modules are specified in the file build.py of the package, "sudo rpm -i ..." commands). In case of problems that you are not able to resolve, please contact slavisa.sarafijanovic@epfl.ch .
FILES:
- README.txt explains what the tool does and what are the steps to use it.
- antispamlab.tar.gz is the full software package of the tool. The configuration file values are already set for testing an example-filter which is also inside the package. The example-filter is a simulated filter that we made just for the purpose that someone can check whether the tool works (well) in his testing environment.
The usage instructions are contained within the README.txt .
Two concrete and detailed examples of use of these instructions to evaluate spam filters are given in the next section.
- Here are the screen printouts from a test we performed on the example-filter (our simulated filter): deployment_log.txt , test_log.txt . When you download the tool, the configuration file values are already set for testing an example-filter which is also inside the package. The example-filter is a simulated filter that we made just for the purpose that someone can check whether the tool works (well) in his testing environment. We supplied the tool with this list of Planet Lab machines: machines.txt . We did run the tool from this Planet Lab machine: master_machine_name.txt . Here is some info about the size and the duration of the performed test: config.txt .
- Here are the screen printouts from a test we performed on the SpamAssassin filter: deployment_log.txt , test_log.txt . The procedure of testing filters other then our simulated example-filter is explained in the README.txt. As an example of this procedure, we provide here the two files that one should prepare in order to test the SpamAssassin filter in the pipe mode: filter.tar.gz , filter_install.sh (antispamlab.tar.gz is the full software package with incorporated filter.tar.gz and filter_install.sh for SpamAssassin). In this test we supplied the tool with this list of Planet Lab machines: machines.txt . We did run the tool from this Planet Lab machine: master_machine_name.txt . Here is some info about the size and the duration of the performed test: config.txt .
- Consistency graphs page shows the results from the multiple tests that we performed using the same filter (we used SpamAssassin), but changing the used set of Planet Lab machines, initial seed or time scale.
To learn more about why did we make this tool, what it offers to you, and how we made it, see this conference paper:
AntispamLab – A Tool for Realistic Evaluation of Email Spam Filters. Slavisa Sarafijanovic, Luis Hernandez, Raphael Naefen, and Jean-Yves Le Boudec. In Proceedings of CEAS 2007, The Fourth Conference on Email and Antispam, Mountain View, California, USA, p. 121-127, August 2007. [Paper-PDF] [Abstract-HTML]
If you would like to (easier) understand the script files of the tool, see the Sections 3 and 7 of this master thesis report [PDF].
Initial work on providing the main components of the tool and putting them to work together in a local environment created within one machine is documented in [PDF].
The current implementation of the AntispamLab provides basic testing functionalities, and we are working to improve its components such are: spam modeling, user behavior modeling, user contacts modeling.
Slavisa Sarafijanovic (slavisa.sarafijanovic@epfl.ch), Luis Hernandez, Naefen Raphael and Jean-Yves Le Boudec
Special thanks to Marc Andre Luthi, for his advices regarding some implementation choices and help in solving some weird "installation doesn't work" cases. We would also like to thank MICS, the main source of funding for the AntispamLab project.
For any questions regarding AntispamLab, write an email to slavisa.sarafijanovic@epfl.ch .