Setting Up a high performance IMAP server

The aim of this article is to describe how to configure a reliable and high performance email server for a 5-100 employees company.

The constraint here is not to send/receive millions of messages a day but to allow users to keep their emails on the server and use IMAP to retrieve them from wherever they are, and for however long they want to keep them.

This type of server is particularly suited for companies whose employees deal with a lot of emails, such as an Engineering, Project Management or a Law firm.

Objectives

The basic objective of this project is to enable authorised users to securely send, retrieve and manage lots of emails from anywhere.

Security is enforced from different angles:

• Authorized use of email for sending, relaying or retrieving is only allowed for users who have an account on that system: for each operation, authentication is required.
• Encrypted communications: whether sending, receiving or managing emails through IMAP, communication between the client and the server are encrypted. This is especially important for roaming users who may use unsecured wireless access points for instance.
• Anti-spam protection to limit the risk of being exposed to phishing scams and to identify good mail from bad.
• Executable attachment protection to ensure users will not receive potentially dangerous attachements in their emails (most executable attachements are forbidden, if you need to send an executable, zip it first). This attachment protection can be extended to control the type of documents sent through emails, for instance, you could ban avi or jpg files based on their extention.
• Anti-virus protection to remove email containing viruses as they arrive at the server. Note that it is recommended to keep an additional level of protection in the form of an up-to-date antivirus programme on each client machine.

Additionally, the technical objectives are:

• enabling users to send any email through the server (with a size limit if you want),
• enabling roaming users to send email through the server securely from the Internet (using encryption),
• enabling users to check their email through a secured web interface, wherever they are (using HTTPS),
• allowing up to hundred of thousands of email messages to be stored in an IMAP folder without noticeable slow down when operating on messages (deleting, reading, getting headers)
• easily add or remove users from the system while not allowing them to access the server from a terminal.
• keeping the system, virus database and spam detection up-to-date

Implementation

For this project, I chose the following implementation:

• postfix will be the Mail Transport Agent, receiving and sending emails and dropping them in each user mail boxes. saslauthd will enable email to be relayed through the server only for users who have an account on the machine.
• dovecot will be our POP3/IMAP server, giving users access to their emails.
• squirrelmail will be our web interface. it uses IMAP to access the local mail boxes.
• amavisd (Amavisd-New) will be our filtering system, marshalling emails beetween postfix and spamassassin -our spam detector- and clamd, the ClamAV antivirus daemon.

Assumptions

In this document, the FQDN? (Fully Qualified Domain Name) of the server is also its hostname mail.example.com and its domain name is thus example.com.

I assume that you have setup a MX DNS record for mail.example.com and that mail sent to joe@example.com will be attempted to be delivered to the server we're setting up.

This setup is meant for a company wanting to host its own email server, so the focus is on single domain only. If you want to manage mupltiple domains, you will have to adapt this setup.

Note: if you need to setup your domain name and take control of your email, I suggest that you take an account with ZoneEdit: the first 5 zones are free, and extra services are quite cheap. You have total control over you domains, subdomains, and can of course create Mail eXchange records. They also have a service to buffer your email should your server be offline. when things return to normal, any email they kept will be delivered as usual.

In the whole insallation and configuration process, I assume that you are logged in under the root account.

Installation

The initial installation process is not critical here. Simply avoid installing too much stuff that you don't need (like X, Gnome or KDE and anything graphical) and limit the number of services that run on the server. It's easy enough to add missing packages afterwards.

Buy your hard-drives in double or triple and mount your partitions as RAID1 or RAID5 arrays: this is quite critical as it means that you won't lose everything if one of the drives fails. I won't discuss backup solutions for this system, but you should have one. In the future, I plan to describe a mail to database solution that could be used as a specific mail backup/archiving/retrieval system.

When deciding your disk partitions, allow a rather large one that you will mount as /mail and format as ext3.
Make it as large as you can. You could also build an LVM on top of that partition if you believe that you could run out of disk space. With LVM, you can easily increase or decrease your available space by simply adding new partitions to the Volume.

During the installation process, make sure that you choose Postfix in the available packages. Fedora defaults to using sendmail, so we'll have to fix that later.

After you've rebooted, use yum to update your system.

Check partition indexes

By default, Fedora Core 4 uses indexes when creating ext3 partitions, this is good and is required for the partition where the emails will be saved.

Why are indexes so important you ask?
Well, since we're going to use maildirs, each email will be saved as a file. Without indexes, it will take an increasing amount of time for the kernel to locate a file in a busy mailbox. The kernel limit that issue by building fast indexes so it can easily find the file it needs.

It doesn't hurt to double check that our partition supports indexes. Let's suppose that your /mail partition is /dev/hda3:

You should see a reference to dir_index.
If not, then you can modify your partition without losing data:

If your system complains about not finding tune2fs, get it from yum install e2fsprogs.

This bit was taken from the Dovecot site.

Users

There are plenty of different ways to authenticate users for mail access.
By default though, Postfix, Dovecot and saslauthd -the services we use here- use the standard unix user accounts to validate users and find out their home directory.

Using LDAP, virtual users, MySQL databases, NIS, SMB or any other scheme is of course possible but in our case I wanted to keep things simple and allow this server to later become say a file server as well without too much hassle. Changing the authentication scheme can be necessary if you're managing virtual email accounts with lots of different domains (if you're and ISP for instance).
In our case, there was nothing much to gain.

Having said that, there are a few of drawbacks to using standard user accounts:

• by default, these accounts are login accounts, meaning that users can log onto the machine at will. While they are not supposed to have enough credentials to wreak havoc in the system, it's still a potential security risk.
• Whenever you create a new user account, its home directory is filled with local configuration files that we don't need (such as bash profiles, or emacs config files)
• The default user account is created in /home but we want them in /mail instead.

I will assume that all our users will not need to login onto the machine by default. If you need such users, you can still override these settings.

Edit the /etc/default/useradd file:

Note: the normal way of doing this is to use adduser -D syntax from the command line, but there is no documented option to change the default skeleton directory.

Now create the skeleton directory:

Now, whenever we use the useradd command, the user will be added to the system without the ability to login and his mail directory will be automatically created.

Note: It's very important that the email folder is not world-accessible: Postfix will otherwise refuse to write any email in it as it would be a security hazard. That's why we set it to chmod 700.

Should you need to add normal login users accounts, you can override the default settings of adduser on the command line as such:

Will add the normal login user susan to the system.

Switch to Postfix

We need to switch from the sendmail, which Fedora uses by default, to postfix.
This stuff is pretty specific to RedHat's way of thinking: they wanted to make email management totally transparent so that it didn't really matter which MTA (Mail Transport Agent) you were using.

To switch, do the following:

The first 2 yum installs may not be necessary if they are already on your system, but it won't hurt if you do it anyway (the -t switch tells yum not to complain if the packages are already installed).

You will be presented with a choice on screen, go for Postfix and press OK. After a few seconds, you should get a succinct report that the switch was completed.

Configuring Postfix

Postfix uses a few configuration files. The most important files are master.cf and main.cf and are located in /etc/postfix.

Let's first make a backup copy of these config files, just in case:

main.cf parameters

Now, edit the main.cf file and update the following definitions. Alternatively, you can use postconf -e "definition" on the command line for each line below if you don't want to edit the file by hand.

This says that mail sent from your server will take the form xxx@example.com.

Note that by default, $myhostname and $mydomain are automatically derived from your machine's name. This name should be a Fully Qualified Domain Name (FQDM) like mail.example.com.
If that's not the case, you can change it editing the /etc/sysconfig/network file or you can declare the myhostname = mail.example.com and mydomain = example.com in the main.cf file itself.

Defines which domains you want to receive mail for. We should always allow the variations of localhost so the server can accept mail sent to itself, and $myhostname and $mydomain ensure that you will get mail sent to both mail.example.com and example.com.
If you want your server to accept mail sent to multiple domains (say myhome.com and loveme.org), add them to the mydestination list as well.

Allows people on the local network to be able to use the server to relay their emails. People from outside the subnet (outside of the IP addresses defined by your network's netmask, such as 255.255.255.0) will not be able to use the server to send email. This is safe, you never want unknown people from the Internet to be able to relay their mail through your server: it would only take a few minutes for your machine to become a spam hub.

Authorises people from the outside to send email that is supposed to be for us.

Defines what sort of information should be sent to the postmaster when there is a problem. There are more options to that, but using too many could flood your mailbox.

Confirms we're not using any external relays as we want the server to deliver our emails directly to other servers. If your ISP doesn't let you send emails by yourself (some block port 25), then you can put their own email server there [mail.isp.com] (including the brackets). Any mail you sent through your server will be given to your ISP's email server for delivery. Note that this is not very reliable as ISP have usually no guarantee that your email will be delivered to its destination: you're in effect sending your mail through a black hole.

Is only needed if your server is not directly connected to the Internet but is for instance behind a firewall that uses Port Forwarding to redirect traffic to it on a local subnet (for instance, your server address is 192.168.0.1 or another reserved LAN IP Class). In that case, you have to tell Postfix what is the outside address of the mail server (replace 1.2.3.4 by whatever is your real IP). Note though that if you don't have a fixed IP, this can be a bit annoying and you may be better off with connecting the server directly to the Internet and using iptables as a good internal firewall.

Makes Postfix listen to all interfaces for email.

Limits the size of emails. Here we set it to 20MB which should be more than enough for most systems. It's a good idea to set a limit. I've have users trying to send 150MB emails to people who only had a dial-up connection (since delivery to the server from the local network is fast, people tend not to notice much the size of the emails they send).

Ensures that mail from other hosts being sent through the server gets rewritten with our domain name correctly appended. this means that if elise@accounts.example.com sends an email through the server, it will be rewritten as elise@example.com.
If you're accepting mail for multiple domains, you must add them to the list as well, separating them by a coma.

Optional and replaces the default name returned by Postfix. It's not a bad idea to replace the default string as it is par of the messages exchanged every time an email is being delivered. Potentially, it could allow someone to use that information to exploit a known security hole (the default string contains the full version number of Postfix).

If that directive is present, it will tell Postfix to deliver messages to the email/ folder inside the user's home, as in /mail/joe/email, instead of the default /var/mail/joe mbox file.
Note that the trailing / means that we want to use maildir instead of mbox. This is how we get to save our received emails in directories and files rather than in one single flat file that becomes cumbersome and fragile if there are too many stored emails in it.

Note: you do not need to create the directories: Postfix will do that for you if they don't exist.

SASL Authentication

As we've discussed before, ensuring that your server is locked down is vital if you don't want to become the next spam relay.
However, we must still ensure that we've got a flexible system that allows all our legitimate users to send email from wherever they are.

SASL is a way of authenticating users when they are trying to send mail. It uses a variety of methods and it's fairly flexible, at the expense of being simple.

To ensure proper SASL authentication, add the following to your main.conf file:

The first line enables authentication for email being sent through the server from the outside (allowing roaming users to send email from the Internet).
the next directive ensures that Postfix will work around older mail clients with broken implementations of SMTP authentication.
Finally, smtpd_recipient_restrictions lists the steps that an email must pass before being accepted. There are a lot of different possibilities. I have used what I think is reasonable for our system.

For security reasons, Postfix runs as an unprivileged user, meaning that it doesn't have access to your password files.
To be able to authenticate users, it must rely on an external service, saslauthd.

Fortunately, this is already installed on Fedora and probably on most distributions as well. Just to be sure, do the following from the prompt:

There is nothing else to do: the default configuration takes care of everything.
The only configuration that tells saslautd that is is needed for mail authentication is in the file /usr/lib/sasl2/smtpd.conf which is installed as part of the Postfix package.
This file contains simply two parameters:

Note: on other Linux systems, this file may be missing or may be located under /usr/local/lib/sasl2/smtpd.conf instead.

Aliases

The minimum alias that must be set-up is for the postmaster who will receive errors and warnings issued by Postfix:

Edit /etc/aliases and modify the following:

The administrator user must have been created and you should probably the one using that account regularly to check for issues.

Note: after every modification of the alias file, you must run postalias /etc/aliases to rebuild the database that Postfix will use. Your changes won't be taken into account if you don't do that!

To add more aliases, just add them to /etc/aliases:

Don't forget to run postalias /etc/aliases to rebuild the aliases database for Postfix.

Virtual map

Aliases are ok, but not very powerful, especially if you're hosting multiple domains and want a mail sent to sales@example.com be redirected to john while a mail to sales@myhome.com should go to suzan.

In that case, edit the /etc/postfix/virtual text file instead of /etc/aliases and add an entry for each possible mail and where you want it to be dropped instead.

Here, anything sent to any variation of Susan's email address will be redirected to Susan's account, regardless of the domain they were sent to: s.smith@example.com and s.smith@myhome.com will be dropped into Susan's mailbox.

For email sent to sales, Suzan will be the recipient for example.com and John for myhome.com.

Any other email sent to invalid addresses (one that does not have a defined account, alias or virtual entry defined on the system) at example.com will be sent to John. This is a catch-all definition, but it is generally preferable to use a separate account for it as it will receive all spam sent to the servers to accounts that do not exist.

After editing /etc/postfix/virtual, you must run postmap /etc/virtual to update the actual binary database used by postfix for fast lookups.

After any change it's also a good idea to run service postfix reload to make sure that postfix will re-read its configuration.

Dovecot for POP3 and IMAP

Dovecot is a good and flexible POP3 and IMAP mail server. It is very performing and can manage large amounts of emails.
Docevot is part of Fedora Core 4 and we don't have to do too much to make it work, although setting encrypted communications takes a bit of work.

If you haven't got it on your system, you can build it from source or just use the ubiquitous yum:

First, we need to tell Dovecot where to fin our emails: as you remember, we told Postfix to use maildirs in the user directories instead of the default mbox.

Edit the /etc/dovecot.conf file and modify the following lines:

Restart Dovecot (service dovecot restart) and test it right away, trying to get email from your test administrator account through POP3 and IMAP. Note that you must have sent email to that account first so Postfix could create the proper directory structure, otherwise you'll get and error when trying to access the IMAP folder that doesn't exist yet.

SSL for IMAP and POP

In the older article for Fedora Core 4 we used stunnel to encrypt our seucred POP and IMAP channels. You can access that article on the SecureAccess page where we still use stunnel for securing email we send to the server..

To change a bit, we're going to follow Dovecot standard configuration and allow it to use SSL directly for POP3S and IMAPS.

First, remove the existing cert files, we're going to replace them later:

Then edit the /etc/pki/dovecot/dovecot-openssl.cnf to look suitatble for your organisation:

Now we need to correct the small script that the dovecot people have nicely provided to generate the key.
Edit /usr/share/doc/dovecot-1.0/examples/mkcert.sh and change the following line:

Now all we need is to run it.

That's it, you should now be able to connect to the server after configuring your client to use SSL.

Installing Amavisd-New

Amavisd is a mail filtering interface that allows an MTA such as Postfix to talk to specialzed external add-ons such as SpamAssassin or ClamAV amongst others. On its own, it doesn't do anything useful, but it will make our anstispam and antivirus tools work well together.

First thing to do is to add a user for running Amavisd-New:

We make sure that the user is unpriviledged and that its home directory is under /var/amavis and to avoid using any skeleton file, we make the home directory ourselves.

Now install Amavisd-New by doing the following from the command line (check the latest version number of Amavisd-New and replace it, this is just an example):

To manage Amavisd-New like any other service, do the following:

Now we can use service amavisd start and stop as with any other service.

Since Amavisd-New is written in Perl and is built aupon the work of many other individuals, we need to install some prerequisites:

The first time you use this command, you will be prompted with a number of questions about your system. Just keep reading and answering, but most -if not all- of the time, just pressing Enter will use an appropriate default.
The only time you need to be careful is when selecting your geographical location and mirrors to download the modules from. Once this is done, things should be smooth.

At the Perl shell prompt, type the long line of prerequisites for Amavisd-New:

You can of course break that into multiple install chunks if you want.

Now we need to make sure that some other optional software needed by Amavisd-New is correclty installed on our machine (they are used to extract content in archived attachments):

Configuring Amavisd-New

Configuring Amavisd-New is simply a matter of editing its /etc/amavisd.conf and change the following:

Here we tell Amavisd-New that it will be running under the amavis user account, that its home directory is /var/amavis and we tell it our domain name as well.

If you are scanning mail for more than one domain, list them in here.

This tells Amavisd-New to always fork 5 children waiting for our mails. This means that, at any time, we'll be able to process 5 emails simultaneously. If you don't have much RAM, you can lower this figure to 2, but you'll have to reflect that as well in the /etc/postfix/master.cf (see below) otherwise Postif will expect 5 processes to be available.

Ensures that we always get X-Spam-Status and X-Spam-Level in our email headers, whatever the spam score, that anything above a spam score (see SpamAssassin) will be flagged as spam.

This is what will be shown in the subject of an email detected as spam.

We will quarantine messages containing viruses so they are not delivered to the user (when our antivirus will be installed).

We also disable virus checking for now because we haven't installed the antivirus yet and Amavisd-New will not let mail through if this option is disabled and there is no Antivirus.

Here we make sure that emails containing attachments with banned extensions get bounced. While we'll probably end up trying to bounce viruses and spam, it's better to use this option to let legitimate senders who are sending us banned attachements that their messages did not get through.

We now have a couple of choices: sending our spam to the user with the modified subject line, or quarantine the message by sending it to a special user account.

Basic setup: spam email sent to user

The most basic setup is to let the user deal with the spam. The good thing about that is that they can judge if there is a false positive, and can still separate the spam by using filters in their mail clients to automatically move marked spam to a local folder.
It is also preferable to do that in a newly setup system as it makes it easier to detect how good your spam catching is and tweak your system.

In /etc/amavisd.conf change the following:

This will make Amavisd-New ignore any action for spam scores below this figure, which is normally beyond the highest possible spam score.

Advanced setup: quarantine to a special account

If instead of cloggin the users with spam messages we want to move them to a special user account on the server, we need to modify the following in /etc/amavisd.conf:

Edit your /etc/aliases and append the following to send all notification emails to the existing administrator user account:

Again, don't forget to rebuild the aliases database with postalias /etc/aliases.

Configuring Postfix to use Amavisd-New

Now we need to let postfix know about Amavisd-New. Postfix communicates with external programs through TCP/IP ports. This method allows a clear separation of processes and their priviledges as each is running under its own account.

Edit the /etc/postfix/master.cf file and append the following:

Note: if you reduced the number of amavisd processes ($max_servers) to launch, you should reflect that number in the first line above as well.

And edit /etc/postfix/main.cf and append this definition:

Amavisd-New White and Black lists

If you want to make sure that some addresses always get through and that some always get banned, you will need to create a white list and a black list.

You need to make Amavisd-New know about those files. Edit /etc/amavisd.conf and add:

Now simply add each email address you want to unconditionnaly allow (white list)/block (black list) on a single line inside the relevant file.

OK, now we're ready to install the antispam and antivirus that will make use of Amavisd-New.

SpamAssassin

Spamassassin is a very versatile and complete spam fighting solution. It uses statistical techniques as well as external blacklist and can be configured to use add-on tools to make its detection more refined.

SpamAssassin uses a rating system whereby each email goes through a list of tests and the mail is flagged for every positive test, increasing the number of spam points it is allocated. Each test allocates a variable number points or fraction of a points depending on how useful and reliable it is at detecting spam.

Once the email has gone through all the tests have been performed an action is taken based on the total number of points: if the score is high enough, we're sure that this is spam, if not, we can let the message through.
There are a number of in-between actions that can be performed, and messages can be refused for deliver, dropped, deleted, quarantined, marked as SPAM in their subject line, etc.

To install SpamAssassin, just use yum:

Edit the /etc/mail/spamassassin/local.cf and add the following:

The last 2 lines are only necessary if your server is using NAT to hide being a firewall and has a local LAN address: you need to tell SpamAssassin about it or it might consider as trusted emails coming from the outside. Change the 1.2.3.4 with your real IP. If your machine receives mail from more than one interface, duplicate those lines for each outside IP.

Now we need to make some changes to ensure that what we do as root is repercuted and accessible to Amavisd-New:

Now, make sure SpamAssassin will run when we boot:

Initialise the Bayesian database:

Test our config by running:

If what you did above was done properly, you should see debug: using "/var/amavis/.spamassassin/user_prefs" for user prefs file in the middle of all those spewed by Amavisd-New (scroll back or use Shift+PageUp keys).

Testing antispam

Just send an email with the following in the body:
XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

This is a standard fake spam signature used to test antispam software.

You should get the email with the [SPAM] in the message, unless you've opted for the other Amavisd-New configuration option of sending spam to quarantine in a dedicated separate user account.

Baysian filtering

Spamassassin includes a powerful statistical analysis that can help toward refining the score given to emails passing through it.
Instead of depending on other people's definition of what is spam, the baysian filter can be trained to recognise what you consider as being spam.

The only drawback with baysian analysis is that it needs a set of good (ham) and bad (spam) emails large enough to be accurate, and you need to sort these emails and manually train spamassassin.
Spamassassin is able to be trained against each user's preferences, but it creates a burden on the user, requiring her to think about and properly pick out any spam and save it in a separate location.
I generally prefer a hand-off approach where users don't have to worry too much about that. One of the reasons being that training spamassassin against an improperly categorised mailbox could actually be counter-productive and allow more spam to be delivered (you always have to think that some users will just delete spam or, worse, leave it in their mailbox).

To avoid any aggravation, I created a simple MissedSpam folder in one of the IMAP mail accounts that I use. I then simply have to move any spam that made its way to an inbox into that folder.
Users can forward their spam as attachments and I have then to manually open and drop the spam into the mailbox. This is a bit time-consuming, but it ensures that I have a chance to evaluate the email and ensure that the spam box is really accurate and does not contain legitimate emails that could later be falsely detected as spam.
As time progresses, less an less spam should be able to make its way into the system.

To train spamassassin as to what is spam and what is ham, make sure you have enough segregated emails (between 150-3000) in each mailbox being trained then issue the following:

That would train spamassassin to recognise spam better.
To function properly (at all) you also need to train for ham. Make sure that each mailbox you train against has no spam in it!

Make sure that the database ownership has not been reclaimed by root:

If you do this regularly the amount of spam that managed to get through should reduce.

Improving spam filtering

SpamAssassin comes with the ability to use custom-rules used to add/remove points to each message passing through.
As a result there are a number of rule lists available that are regularly maintained to contain the filtering rules against the latest spam and other more serious threats.

On interesting project is Rules Du Jour at Exit0Wiki, a SpamAssassin-dedicated site.
It is in fact a script that allows you -using a configuration file- to download an install a lot of rule files from various sources without having to worry about manually deploying them yourself.

Installation

To install Rules Du Jour is simple enough and here is a quick recap.

This will have downloaded the script and made it executable by root and created the blank configuration file that we now need to fill.

My config file contains all available rules. Just remove the ones you don't need and don't forget to change the notification email address to something suitable to your site.

Now we just need to make this run daily (please only run once per day to avoid stealing too much bandwidth from the sites that kindly give us these free goodies).
Update your crontab y issuing crontab -e on the command line and appending the following on a new line:

Change the time to something else to even-out the load on the servers that we will download from.

Now the last thing we can do is try to run the script. It will output a lot of info as each rule is downloaded and installed.
Once all are done, spamassassin will test them and if no errors are reported, will restart itself.

You should now have another, improved layer of email protection.

ClamAV Antivirus

Clam AntiVirus is a totally free -in all senses of the word- antivirus scanner. It is released under the GPL and has seen a lot of activity in the recent past.
Its virus definitions are generally good and new virus definitions can be automatically downloaded several times a day.

To install ClamAV, I recommend installing from source or getting the binary from the Crash-Hat yum repository.

Download the latest source from http://www.clamav.net/stable.php then:

Note: you may need to upgrade the zlib library to the latest version if the configure step complains about it.

Edit /etc/clamd.conf and add/modify the following parameters:

All other parameters should be commented, in particular Example and TCPSocket.

Now we need to tell Amavisd-New to enable virus scanning. Edit /etc/amavisd.conf and comment out the following :

Look for the following section in /etc/amavisd.conf and uncomment it:

Automated update of Virus definitions

Edit the /etc/freshclam.conf:

Now create a new crontab -e job to launch the updater:

Now freshclam will check for updates five minutes past every hour.

Note: Make sure that the name of the socket file clamd.sock matches the one used by clamd above!

Testing the AntiVirus

Start our AntiVirus service and restart our setup to check if everything works fine.

Note: Postfix should always start after amavisd!

Send an email with the following in the message body (in an HTML email) or as an attachment (in a text file if you are sending the email as plain text only):
X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

This is a recognised fake virus signature that should trigger the ClamAV. That message should normally only be received by the account we set up in /etc/amavisd.conf. See AmavisdNew for the configuration options. Note: plain text message bodies are not scaned for viruses as they are safe, only attachments are scanned for threats. On the other hand, HTML message bodies are scanned and treated as attachments.

SquirrelMail web interface

SquirrelMail is the webmail interface that comes preloaded with Fedora.
If it's not installed, just use yum install squirelmail to get it. You will also of course need Apache and make sure that your firewall has port 80 (http) and 443 (https) open.

Make sure Apache is running service httpd start. SquirrelMail is available out of the box from http://mail.example.com/webmail/.

By default, SquirrelMail is not accessible from https, and since our server will only be for mail, there is not need for appending the /mail at the end of the URL, so we need to fix those shortcomings.

First get the original SquirrelMail Apache config file out of the way:

Then edit the /etc/httpd/conf.d/ssl.conf and add the following between the existing <Virtualhost> tags, then append the URL rewrite rules:

Now make sure that Apache will start at boot and start the service for now:

You should be able to access your email from anywhere securely through http://mail.example.com/.
Requests to the web server wil be transformed as such :

• http://mail.example.com
  -> https://mail.example.com
• webmail.example.com (whether http or https)
  -> https://mail.example.com

Note: https connections require a digital certificate registered with a known authority. The certificate is only valid for one website and one IP address and you need to pay for it. If you don;t have a certificate, your users will receive a warning when trying to access the site. You will have to tell them not to worry about that if you don't want or can't have a certificate (if you're using dynamic IP for instance). The certificate is only necessary to confirms that the site using is really who it pretends to be, it doesn't affect the fact that communications are encrypted.

Firewall considerations

Email services by definition need to be able to connect to the outside world. Unless you are content being able o exchange emails in you own little private LAN, it's likely that -in the minimum configuration- at least your SMTP server will be able to access the outside world to send emails.
A more realistic configuration -as the one we've been building here- would allow access to roaming users getting their emails from outside the private LAN and would allow messages to delivered to our SMTP server, postfix.

The following TCP ports are those used by mail services:

• SMTP: 25 (Plain text transfers)
• IMAP: 143 (plain text transfers)
• POP3: 110 (plain text transfers)
• SMTPS: 465 (Secure, SMTP over SSL)
• IMAPS: 993 (Secure, IMAP over SSL)
• POP3S: 995 (Secure, POP over SSL)
• HTTP: 80 (For for webmail, plain text transfer)
• HTTPS: 443 (For for webmail, Secure HTTP over SSL)

Now, depending on how your users are supposed to access their email, you will need to open the necessary incoming POP, IMAP and HTTP ports and/or their secured variants on your Internet-facing connection.
In any case, you will need to open port 25 so other servers can communicate with yours and deliver their messages. If you want roaming users to be able to deliver their messages securely through your server, port 465 must also be opened.

In a next chapter, we will see that it is fairly easy to secure your email channels using SSL and circumvent any port blocking that often ISP put in place.

Firewall implementation

Having a proper firewall on your server is a must. Instead of relying on the default firewall configuration, I always opt to use the strong firewall rules scripts that allow better security and monitoring of the connections.

These rules scripts are available in the IP-Masquerade-HOWTO available from The Linux Documentation Project site.

Specific rules

In my firewall script, I use the following rules in the OUTPUT section of the script:

If you are not using the same firewall script, replace the following variables:

• $IPTABLES by /sbin/iptables
• $EXTIF by your Internet-facing interface (eth0 for instance)
• $EXTIP by your Internet IP address.
• $UNIVERSE by 0.0.0.0

To ensure that the script is invoked at startup and whenever my internet connection drops, I saved the script as /etc/rc.d/rc.firewall and added that path to my /etc/rc.d/rc.local startup file and my /etc/ppp/ip-up.local file which is executed whenever the PPP connection restarts.

Make sure the script is executable and modifiable only to root and restart it:

That's all you should need to make your email services accessible from the outside world without compromising your whole machine.
Remember that you don;t have to poke all these holes in your firewall; only open the ports for the services you really need.

Secure, encrypted email channels

With the proliferation of wireless hotpots, people have been able to enjoy a new freedom in accessing the web and their emails: airport lounges, coffee shops, public spaces.
While all this freedom is wonderful, public access points (and most private ones) usually don't use any form of encryption to protect your transfers.
This is especially an issue on publicly accessible networks where absolutely anyone can eavesdrop on your communication without you being the wiser. In some countries (including the US it seems), internet communications are routinely filtered, taped or analysed for whatever purpose is deemed just, too often without any accountability.

One of the reasons that email can be the weak link in your organisation security is that most of the time authentication connections by your email client are still sent in clear text, meaning that every time you get email, you advertise your username and password.
A lot of organisations only use a single username/password for each of their users and their access needs, meaning that the same pair is not only used for authorizing access to email accounts, but also to login into your PC, domain or Intranet...

Stunnel to the rescue!

There are many ways to secure your email connections through SSL: most servers have their own configuration details allowing them to work through an encrypted channel.

Here though, will will use Stunnel to provide secured channels to postfix@2 and dovecot@@ rather than configuring each of them individually.

Stunnel is also easy to use and can allow you to create ad-hoc secure tunnels for any arbitrary network service as well.

Installing Stunnel

If Stunnel isn't on your machine yet (whereis stunnel), just grab it from your rpm repository (yum install stunnel) or get the source code from stunnel.org.

To install the source code:

Note that if you are installing on a RedHat machine, you will need to modify the files tools/Makefile.am and tools/Makefile.in to replace any occurence of nogroup with nobody instead.

Note as well that the default installation directory is /usr/local/bin. If you don't want this, use the --prefix option of configure (call it with --help to get more details about the available options).

Creating the private key and certificate

To be able to encrypt the communication, Stunnel needs to have access to a secure and unique private key and a certificate that it can issue to the clients connecting to it.
the certificate will contain information about your organisation and a public key that will be used by to encrypt data that can only be decrypted by the server running stunnel.

This step is important: don't use any default key that comes with your system: the communication channels would still be encrypted, but since these keys are publicly available, your communications could be decrypted without too much hassle.

To create the necessary keys just do the following:

You will be asked a number of questions regarding your organisation. These will be included in the certificate issued by Stunnel when contacted by clients.
Also make sure you use the FQDN (Fully Qualitified Domain Name) of your email server when asked about your server name (for instance, mail.example.com).

Note that the certificate will only be available for the number of days specified when creating it. You can make your certificate valid for as little or as long as you wish.

Now, Stunnel requires that both certificate be in a single file with extra blank lines after each certificate.
Like most encryption-related configurations, it also requires that the certificate have limited access.
After creating the proper key, we don;t need the original ones any longer.

That's it, Stunnel is ready to be configured:

Create the /etc/stunnel/stunnel.conf with the following entries:

What this does is simply tell stunnel to listen to ports 993, 995 and 465 and spit out the decrypted stream of data to ports 143, 110 and 25.
The cert line should not be necessary unless you compiled stunnel from source and it is expecting a different path (check what it could be by running stunnel version).

Now we can try to launch stunnel manually to check that it works:

That's all that should be needed. If you are getting error messages, then try to add the path to your stunnel.conf file. If there are any other errors, we won't see them and stunnel won't tell us whether eveything is fine or not.
To make sure everything is OK, have a look at the /var/log/secure log file, it should end with the following lines:

If you get an error saying that the binding address is already in use check that your protocols entry in the /etc/dovecot.conf file doesn't contain pop3s or imaps.

To Allow Stunnel to become a normal service like any other, we're going to install an init script for it.

A script is provided in the source code tree under tools/stunnel.init but I recommend you get my copy instead as I have modified it to be manageable with the chkconfig and service utilities.

Install the service and schedule it to run:

That's should be all. You can check that stunnel will start and stop by using service stunnel start or service stunnel stop.

Alternate Access

It is not uncommon for ISP to deal with the problem of spam and other email annoyances by forcing users to go through their own email services.
Most often, they will block the SMTP port 25, barring your roaming or external users or offices to directly connect to your mail server to deliver their mail.

Another species of more devious filtering seems to be common in places like China. The largest provider, China Telecom, seems to have strange ways of limiting (or filtering) access to IMAP and POP services when connecting to servers outside of China.
I'm not sure how and why, but access to these ports would be OK for w while, then would suddenly be blocked while all other traffic would still be available.

To solve these issues I'm discussing here 2 solutions:

• opening alternate ports to our SMTP or POP and IMAP services.
• offering alternative encrypted channels for STMP or POP and IMAP services.

The first one may be enough to circumvent the restrictions imposed by most ISP and will work fine as long as their filter only block the usual email ports 25, 110 or 143.
We use IP forwarding to poke a hole in the firewall and redirect any incoming traffic to the standard ports.

The second also requires that we poke holes in our firewall, but we'll just ask stunnel to listen to other ports as well as the standard SSL ones discussed in the SecureAccess article.

Forwarding traffic from one port to another

So let's implement the first solution.
I'll chose to arbitrarily use ports 725, 710 and 743 for SMTP, POP and IMAP.

As discussed in the Firewall chapter, I use the strong firewall rules scripts provided by the Linux IP Masquerade HOWTO.

In the FORWARD section of the script, I add the following entries:

Now restart the script (should be saved as /etc/rc.d/rc.firewall) and modify your email client to send and receive its mail using the new ports.

Secure alternate access

Again, we're going to use alternate ports 825, 810 and 843 as our arbitrary secure ports.

First, we poke holes for these ports in our firewall, so just edit /etc/rc.d/rc.firewall and add the following to the OUTPUT section of the script:

Restart the script and add the following entries in the the stunnel configuration file /etc/stunnel/stunnel.conf discussed in the SecureAccess chapter:

Restart stunnel (service stunnel restart if you installed the init script for it) and modify your email clients to use SSL and the alternative ports.

Mbox to Maildir migration

When upgrading a mail server, you may be faced with a rather 'interesting' problem: migrating old user account in the classic monolithic mbox format to the more tolerant and performing maildir.

Mbox is a way to store emails in a single file. It's old, compatible with most mail applications, but as your mail store grows, it becomes harder to manage emails contained within.
Things start to become shaky if your mailbox is large, with the risk of corruption or connection time-out becoming likely.

Maildir on the other hand is newer and not supported by all systems. Basically it's just a way to store emails as files within a folder hierarchy. Its advantages are mostly better performance and tolerance in case of failure since you're more likely to lose just one email instead of your entire mailbox.

Postfix (MTA) and Dovecot (POP3 and IMAP server) both support the maildir format and can integrate with each other seamlessly.

Migration

It's nevertheless a great pain when you have to migrate existing mailboxes to the new format, on a new server for instance.

There is apparently only one tool that allows this type of thing: it's a small Perl script maintained by Philip Mak that is available as Perfect_maildir.

Just download the single file script and you are ready to convert, unless you're getting a long error message and in that case you may need to download the Date::Parse Perl module.

To install it, just enter:

If this is the first time you install a module, you will be asked a series of question. Just keep pressing Enter until you're asked about your geographic area. Make your selection of mirrors to download from and Perl will do the rest. Next time, you won't be asked anything.

If you have a handful of accounts to migrate, doing it by hand may be tedious, but it can still be faster than going through setting up an automated process.

I found useful to use my mail client to create IMAP accounts for every user I needed to migrate, then use their original mailbox layout to re-create the directories they had.

You can get the old mailbox layout by viewing the .mailboxlist file in the user's home.
Once each directory had been re-created (using your mail client), use perfect_maildir.pl to migrate the email for each folder, for instance:

Would first move the inbox and then the old Draft mboxes to the new maildirs and ensure that the files belong to the user; this is particularly important if you're doing the migration as root.

If you need to migrate a lot of accounts, then you should use the migration tools below. They make use of perfect_maildir.pl, so you'll need that as well. I've included a copy of the script on this site just in case.

Just in case it may be useful to you, here is a small script I used to move each original mbox inbox to the new maildir one:

You will need to modify the hard-coded paths to suit your configuration. In mine, I used the following:

• /mail/<username>/email is the destination maildir folder.
• /oldmail/spool/<username> is the old inbox mbox.

If you're setting up a new machine and your inboxes are on the old one, you can make files available to the new system through NFS and mounting the exported directories locally or simply copying using scp.

Alternatives

If your old email is sitting in a POP3 account at an ISP, you can use fetchmail to retrieve it.

Fetchmail is easy to use and I won't go into the details except that you could use a /root/.fetchmailrc file containing one line per account for each user. Note that this file must be chmod 600 for security reasons or fetchmail won't touch it:

The keep at the end will leave the mail on the old account just in case.

Another way, useful if you are using IMAP on the old account and have created IMAP folders, is to have both old and new account in your client email for each user and drag-drop files from one to the other after having recreated the same directory structure manually.

Email clients configuration

...ToDo...

Troubleshooting SASL authentication

To ensure that your authentication process works fine, we'll check what the server reports when we try to feed it a correct login.

Since logins are base64 encoded, copy and paste the following in a file that you call encode_sasl_plain.pl (don't forget to chmod 755 to make it executable):

Then use it to encode a username/password pair as it would be expected by the mail server for authentication. Here I use the existing administrator user (the account must exist on the system):

Then, talk to your mail server manually:

The only lines you will need to type are 6, 15, 17. The others are the server's responses.

Maillog analysis with AWStats

Maillogs are mighty important when troubleshooting or to check if your server is properly configured for its load.
The problem with raw maillog is that it quickly becomes hard to get any time of big picture view of what's going on: you can easily trace individual connections, but you won't know how many emails you send every day, how many megabytes are being moved or how many DNS errors your system is having.

AWStats is a generic log analysis package that offers a nice web interface and can be made to analyse almost any type of logs. It's easy to extend too and quite simple to install.

I don;t recommend using the RPM from the AWStat website has it has been packaged differently for Fedora and if you use automatic update, your config will change. Just use yum to get it and make a virgin configuration file:

The second line creates an alias for the awstats_updateall.pl to include the path to the awstats.pl script. This is necessary as the Fedora version of AWStats moved the files to other locations but some scripts still expect them to be at the default one.

Then edit the /etc/awstats/awstats.mail.conf file (don't forget to change the mail.example.com to your server hostname):

Now make AWStats update its database regularly from a cron job (crontab -e), for instance, every 2h for mail analysis and every 3h for the web logs analysis:

Then we can construct the database right now so we can access it for the remaining of our configuration:

If there are errors, double-check that you entered all the correct options in the configuration file above. If the script says it can't find awstats.pl, make sure your ~/.bashrc file contains the alias we defined above (there should be no spaces on either side or the = sign) and that you have logged off and back in to ensure the aliases were read (the ~/.bashrc file is read each time you login).

AWStats Integration in Apache

So now we have AWStats installed and updating its database regularly but what we still need is to display the stats in a convenient way.

The default installation should have added the following lines to your /etc/httpd/conf.d/awstats.conf file. to make this set up a bit more secure, we're going to allow access only from web browsers on our LAN and maybe from our other office at 212.87.250.3, all others will get an Access Denied page instead:

Now, to access AWStats, you need to point your web browser to http://mail.example.com/awstats/awstats.pl?config=mail .
However, if you configured SquirrelMail in a similar way as we did, this will probably not work as we have been rewritting our web requests to ensure people were using HTTPS.

Our solution, amongst many possible, was to add a subdomain mailstats.example.com pointing to our mail server and add the following rewrite rules in the /etc/httpd/conf.d/awstats.conf :

Requests to the web server wil be transformed as such :

• http://mailstats.example.com or http://mailstats.example.com/mail
  -> http://mailstats.example.com/awstats/awstats.pl?config=mail
• http://mailstats.example.com/web
  -> http://mailstats.example.com/awstats/awstats.pl?config=localhost.localdomain

Now, providing we created the mailstats sub-domain, we can access both our mail and web statistics without having to remember those long URLs.

Extending AWStats

AWStats by default will provide quite a bit of useful information, but it will also ignore some that it won't understand, like how much spam or viruses were found.

How to integrate this information into the stats collected by AWStats will be developped in the near future.

References

• How to install Postfix, Amavisd-new, SpamAssassin, Pyzor, Razor, DCC, and ClamAV on Fedora Core 4 - v2.1
• Switching To Postfix in Fedora
• Apache config for Squirrelmail

• http://www.ijs.si/software/amavisd/

• SpamAssassin Project

• The Rules du Jour website

• Clam AntiVirus, a GPL virus scanner
• Crash-Hat yum repository for latest ClamAV binaries.

• Stronger firewall rulesets
• http://www.iptables.org/

• http://www.stunnel.org/
• Using stunnel to Encrypt Communications to Clients
• http://www.openssl.org/docs/HOWTO/

• Replacing UW-IMAP with Dovecot on Red Hat Enterprise Linux 3 and its migrate.tar tools.
• Perfect_maildir original site.
• perfect_maildir.zip, the actual script, just in case it is not available from above.

• http://awstats.sourceforge.net/ : AWStats home
• http://awstats.sourceforge.net/docs/awstats_faq.html#MAIL : AWStats configuration for maillogs
• http://httpd.apache.org/docs/2.0/vhosts/ : Apache virtual hosts
• http://httpd.apache.org/docs/2.0/rewrite/ : Apache's Mod-rewrite documentation