E-mail server section

E-mail server setup

What we want to accomplish here is the setup of a mail server with the following properties:

• can serve multiple mail domains
• can relay mail for other domains to other mail servers
• can have one or more mailboxes per domain
• users of these mailboxes can be virtual (do not need to have a Linux user account)
• can have multiple aliases per mailbox
• can forward mail for certain aliases to multiple mailboxes

For this type of mail server setup, we owe a great thankyou to Christoph Haas, whose advise has helped us create flexible and reliable mail servers since 2003.

Preparation

We'll assume that the server currently has no mailserver installed, at least no other than the default exim mailserver. Furthermore, the server is already fitted with MySQL, and this database is running without problems.

The hostname of the server must be set correctly, so that hostname -f returns a valid DNS name, like lighthouse.saruman.biz.. It might also be an internal name like lighthouse.saruman.lan. but that will require us to give extra attention to the name under which Postfix will contact its collegues on the Internet. Also, the server can correctly [Networking_section#DNS_resolution_under_Debian | resolve DNS names] like www.debian.org, preferably by running it's own caching DNS server.

The server is kept on the correct date and time using NTP, TCP port 25 is open on the server, the ISP will allow connections from Internet to this port, and if there's a firewall running on this server, then it has port 25 open so as to not block incoming e-mail.

Software installation

As a first step, we use apt or aptitude to make sure that our server is up-to-date. Then we can install the necessary software packages. Under Debian 5.0 "Lenny", the (single) packages is:

• postfix, the mail server itself - this includes the "virtual package" postfix-tls, that we want to use to secure connections to Postfix with the TLS protocol

At the same time we can - and must - purge the following packages:

• exim4
• exim4-base
• exim4-daemon-light
• exim4-config

In aptitude, only press "go" when you've marked all four of these packages "purge", or you cannot install postfix.

When installing the postfix package, the Debian installer script will ask you several questions, which you can answer like this:

• General type of mail configuration: Internet site
• System mail name: the FQDN of the mail server that you've verified in the previous section. Note that the script will try to guess the DNS name, but that might yield a DNS name with a trailing dot. That is technically correct, but the installation script will barf. Remove the trailing dot before hitting <enter>!
• Postmaster mail address: the address that all mail should go to that is addressed to "root" and "postmaster".
• Domain list: give the list of all domains that the machine can consider itself the FINAL destination for. This should at a minimum include an empty value, "localhost" and the FQDN of the machine itself (no trailing dots!); however, if you're running your own mail domain, you can also add that (e.g. "saruman.biz"). Thus, the list could look like this:

saruman.biz, lighthouse.saruman.lan, localhost.saruman.lan, , localhost

• Force synchronous updates? We think that's not necessary, but please read the question and decide for yourself
• Local networks: something like 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.67.0/24 (the default, augmented with your local IP range)
• Mailbox size limits: you can give postfix a limit in bytes, but we're going to use one single big mailbox for all users, so we cannot let Postfix guard it. Leave it at 0 (zero) so we don't have a size limit.
• Character for local address extension: we leave it at +
• Internet protocols to use: currently we don't have IPv6 support, so there's no sense in letting Postfix try to serve IPv6. We choose ipv4 only.

With the above data, the Debian install script for Postfix can do its job and configure Postfix with some basic settings.

Now that Postfix is installed, we can install some dependent packages (we could install them in the same run, but if anything goes amiss with the postfix installation, then these packages are going to remain unconfigured as well):

• postfix-doc, the accompanying documentation;
• postfix-mysql, necessary to have Postfix talk to our MySQL server;
• postfix-pcre to be able to parse regular expressions, which which we can combat spam;
• dovecot-imapd is a daemon that will provide your users with IMAP access to their mail;
• dovecot-pop3d is another daemon, but for the POP3 protocol.

Virtual Mailman creation

When we're done, we'll need a "system user", a sort of virtual mailman that is the owner of all mailboxes that we're serving. We suggest the name "vmail" for this user. Note: this user does not get his own mailbox (i.e. there's no mailbox at vmail@saruman.biz).

To create this user, and his home directory, we can run the following commands:

groupadd -g 120 vmail
useradd -g vmail -u 120 vmail -d /data/vmail -m -s /bin/false

As you see, we've chosen a group ID and user ID of 120 (after confirming that this ID was not taken by another group or user, by checking /etc/passwd and /etc/group. Furthermore, we've decided to keep the vmail user's home directory not under /home/vmail, but in a special place where we're going to expect server data to reside - in our server, the /data directory (which is a RAID-5 array mounted under root). By adding -m, we've actually created the home directory, and adding -s /bin/false makes totally sure that the user vmail can never ever log in - even if we've not created a password for this user, so vmail shouldn't be able to log in anyway. Better safe than sorry.

To tell Postfix that this vmail user is someone special, we run

postconf -e virtual_uid_maps=static:120
postconf -e virtual_gid_maps=static:120

That makes postfix understand that all mail for the virtual mail domains must be written to disk with these specified user and group ID's.

MySQL configuration

Database preparation

We will use the MySQL database to record data on our mail system, and then give our Postfix access to this database so that it can read its configuration from there. For starters, we'll create a MySQL database named "vmail", and a MySQL user named "vmail_admin" that can read all necessary data from that "vmail" database. Then, we create the necessary tables, and a view that links these tables. We do this with the MySQL client mysql. However, we're quite lazy, so we don't create this database by hand (that's error-prone), but by use of a script create.vmail.sql. To this end, feed the create.vmail.sql script into the mysql client like this:

mysql -u root -p < create.vmail.sql

(This of course assumes you have create.vmail.sql in your current working directory; if not you can include the path to the file.) Simply give the MySQL root user password, and the script creates the database, the user, the necessary tables, and the view.
A note of caution: it is never a good idea to just run scripts without a proper understanding of what it does. Especially with MySQL, it will be advantageous if you understand the SQL commands. Open the script in a text editor, open the MySQL command reference, and trace back what the script does exactly.

Inserting data

Next up, we fill the database with the first sets of values (either test data or the first of our production data). We'll start off with the virtual domains that we're hosting, by running the mysql client and feeding it information like this:

mysql -u root -p vmail
mysql> INSERT INTO virtual_domains (id, vdomain) VALUES (1, 'saruman.biz');
mysql> INSERT INTO virtual_domains (vdomain) VALUES ('wiki.saruman.biz');
mysql> INSERT INTO virtual_domains (vdomain) VALUES ('shop.saruman.biz');

This has the effect of creating three entries. You can check that everything worked as planned by executing

mysql> select * from virtual_domains;

Note: only the first entry needs an id value, because in MySQL we've defined that field as AUTO_INCREMENT. After creating your first virtual domain in the table, you never have to use a statement like the first INSERT again, only statements like the other two.

Now the MySQL database has the information needed by Postfix to recognise that you have three virtual mail domains (namely the three domains in the VALUES section) for which it hosts virtual mailboxes. Postfix cannot read this information yet, but that'll be taken care of in the next section.

Whle still within the mysql client, we can now create users:

mysql> INSERT INTO virtual_users (id, domain_id, user, passwd) VALUES (1, 1, 'jan', MD5('JanSecret'));
mysql> INSERT INTO virtual_users (domain_id, user, passwd) VALUES (1, 'mike', MD5('MikeSecret'));
mysql> INSERT INTO virtual_users (domain_id, user, passwd) VALUES (3, 'shopkeeper', MD5('ShopSecret'));

This has the effect of creating three users "jan" (in domain saruman.biz), "mike" (in domain saruman.biz) and "shopkeeper" (in domain shop.saruman.biz). Again, the id value is only ever needed in the first statement, because from now on every user addition will auto-increment id.

The passwords shown are encrypted with MD5, and put in the passwd field. Later on, the users will be able to access their mailboxes using this password; we won't tell Postfix anything about them, because it doesn't need the passwords. You can check the content of the virtual_users table with the right "select" statement, and you can now also see that the view view-users works:

mysql> select * from virtual_users;
mysql> select * from view_users;

Should you need to update a user's password from the command line, you can use the following command, provided you've looked up that user's id :

UPDATE virtual_users set passwd = MD5('<password>') WHERE id = '<id>';

Next up, we're going to add some aliases, which are alternative e-mail addresses for the users; their primary e-mail address can already be seen in the view_users view, but perhaps you also want mail to "webmaster@shop.saruman.biz" to arrive at one or more mailboxes, e.g. in the mailbox for user "shopkeeper" (within domain "shop.saruman.biz") and this guy's home address ("j.doe@example.com"). Furthermore, we'll define catchall-addresses for all domains, that'll send all mail for which no mailbox can be found to the mailbox of user Mike (for the first two domains) and Webmaster (for shop.saruman.biz; Webmaster himself is an alias yet again, that points to shopkeeper@shop.saruman.biz). To create the catchall, leave out a value for the source address. This creates the empty value for source that will be expanded (using source@domain) to "@domain".

mysql> INSERT INTO virtual_aliases (id, domain_id, source, destination)
       VALUES (1, 2, 'webmaster', 'shopkeeper@shop.saruman.biz');
mysql> INSERT INTO virtual_aliases (domain_id, source, destination)
       VALUES (2, 'webmaster', 'j.doe@example.com');
mysql> INSERT INTO virtual_aliases (domain_id, destination)
       VALUES (1, 'mike@saruman.biz');
mysql> INSERT INTO virtual_aliases (domain_id, destination)
       VALUES (2, 'mike@saruman.biz');
mysql> INSERT INTO virtual_aliases (domain_id, destination)
       VALUES (3, 'webmaster@saruman.biz');

Again, we don't need to add the id value any more after the first ever insertion into this table.

We can check the result by running a select statement against the virtual_aliases table and view_aliases view:

mysql> select * from virtual_aliases;
mysql> select * from view_aliases;

Postfix configuration for MySQL lookups

Next, we're going to tell Postfix to use the vmail database, and also how to read the database (Postfix never writes the MySQL database). To this end, we're going to create three configuration files in directory /etc/postfix. We'll start off with one configuration file, with which Postfix can determine if a domain name is among the domain(s) that it's actually hosting mailboxes for. Then we'll create the config file that checks the table that contains all the users that have a virtual mailbox, and finally we create the lookup for the table with all the aliases.

Virtual mail domains lookup

Create file /etc/postfix/mysql-virtual-domains.cf with the following content:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT 1 FROM virtual_domains WHERE vdomain='%s'

Next, we tell postfix to check this configuration file when it needs to check "virtual_mailbox_domains":

postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-domains.cf

Use of this configuration file in postfix has the effect of returning "yes" when checking our database for the domain part of an email address. Naturally, this configuration file has to be fitted with the actual vmail_admin password rather than our example "SuperSecret".

Virtual mail user lookup

Create the file /etc/postfix/mysql-virtual-mailbox-maps.cf with the following content:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT 1 FROM view_users WHERE email='%s'

Next, we tell Postfix that this mapping file is supposed to be used for the virtual_mailbox_maps mapping:

postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

The lookup of virtual mailboxes should work with the data we've put into the database previously. i.e.

postmap -q jan@saruman.biz mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

should return a 1 to acknowledge that "jan@saruman.biz" is indeed a virtual mailbox in our Postfix configuration.

Virtual alias lookup

Create another cf file at /etc/postfix/mysql-virtual-alias-maps.cf:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT destination FROM view_aliases WHERE email='%s'

And create yet another cf file at /etc/postfix/mysql-email2email.cf:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT email FROM view_users WHERE email='%s'

Again, we tell Postfix that this mapping file is supposed to be used for the virtual_alias_maps mapping:

postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf

The lookup of virtual aliases should now work as designed:

postmap -q shopkeeper@shop.saruman.biz mysql:/etc/postfix/mysql-virtual-alias-maps.cf
postmap -q webmaster@shop.saruman.biz mysql:/etc/postfix/mysql-virtual-alias-maps.cf

If you've put in the data from the previous section, then both commands should return the same result: shopkeeper@saruman.biz. This shows that Postfix will deliver mail to shopkeeper or to webmaster to the mailbox of shopkeeper.

Finishing up configuration files

When everything works as planned, then we secure the configuration files against prying eyes. Remember, the configuration files contain the user/password combination which which to access the vmail SQL database. The vmail_admin user has only read rights, and passwords in there are encrypted, but nevertheless, this is information we need to protect. Thus:

chown root:postfix /etc/postfix/mysql-*.cf
chmod u=rw,g=r,o= /etc/postfix/mysql-*.cf

This protects all four configuration files since only root can write them, members of group postfix can read them, and the rest of the world cannot access them at all.

Configuring mail delivery through Dovecot LDA

E-mails get delivered to the virtual mailboxes by means of a Mail Delivery Agent (MDA). With Postfix, the standard MDA for delivery to virtual mailboxes is called virtual, but we're not going to use that; we're going to use Dovecot LDA, which is a program called deliver. By the way, the abbreviation LDA stands for Local Delivery Agent, which of course means more or less the same as MDA.

Enabling the Dovecot daemon

To make Postfix use Dovecot LDA, we add the following line to /etc/postfix/master.cf:

dovecot   unix  -       n       n       -       -       pipe
    flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient}

This declares the service "dovecot" to mean using the program /usr/lib/dovecot/deliver with the right parameters. If we now let Postfix reload its files, it'll learn about this new service. Subsequently we can instruct Postfix to start using it, by declaring service "dovecot" to be the virtual transport of choice, and specifying one Dovecot-specific variable:

postfix reload
postconf -e virtual_transport=dovecot
postconf -e dovecot_destination_recipient_limit=1

Dovecot default configuration

Now to configure Dovecot itself: open /etc/dovecot/dovecot.conf with your favourite editor vim; first make sure the protocols = line reads

protocols = imap imaps pop3 pop3s

(which it should by default). The second setting is quite crucial. By default, the location of the mail directories is found automatically by Dovecot, but here that won't work. Find the line that reads #mail_location =, and change it to

mail_location = maildir:/data/vmail/%d/%n/Maildir

This will look for mail in directory /data/vmail/DOMAIN/USER/Maildir, and expect this directory to be in "maildir" format. Next look for a section called "auth default". First define the allowed authentication mechanisms:

mechanisms = plain login

Then inside that same section you need to uncomment:

passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
}

which tells Dovecot that the passwords are stored in an SQL database (it's obvious that we'll now have to create this conf-file) and:

userdb static {
    args = uid=120 gid=120 home=/data/vmail/%d/%n allow_all_users=yes
}

to tell Dovecot where the mailboxes are located. This is similar to the mail_location setting. Next task: comment out the section passdb pam, so that Dovecot doesn't accidentally try to service local users as well.

Next up: tell Dovecot to handle IMAP the same as previous versions of this virtual mailserver have - this can prove to be quite tricky if you already have mailboxes full of mail, put there by e.g. Courier-IMAP. What works for us, is

namespace private {
    separator = .
    prefix = 
    inbox = yes
}

However, your mileage may vary.

You'll also want to comment out the passdb pam sections, since we're not hosting mail for local users.

Next up, we'll edit the section socket listen so that Dovecot can access the user database (master section), and our Postfix mailserver can access Dovecot (client section):

socket listen {
  master {
    path = /var/run/dovecot/auth-master
    mode = 0600
    user = vmail
  }
  client {
    path = /var/spool/postfix/private/auth
    mode = 0660
    user = postfix
    group = postfix
  }
}

Finally, the protocol section, where we specify log files, an address to put in mails when we send out nondelivery reports, and a place to store scripts:

protocol lda {
    log_path = /var/log/appslog/mail/dovecot-deliver.log
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = postmaster@saruman.biz
    mail_plugins = cmusieve
    global_script_path = /data/vmail/globalsieverc
}

You could change the log path to "log_path =" With a empty logpath syslog will log the events.

Now that we've finished the Dovecot configuration file, we need to create a fitting MySQL configuration file for Dovecot. The file is /etc/dovecot/dovecot-sql.conf, and its contents are

driver = mysql
connect = host=127.0.0.1 dbname=vmail user=vmail_admin password=SuperSecret
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, passwd as password FROM view_users WHERE email='%u';

When you've created this file (as root), change the attributes of dovecot-sql.conf and dovecot.conf, and then restart Dovecot to start using the new configuration. Note: dovecot.conf needs to be read by Postfix, hence the group ownership for the file.

chmod u=rw,g=,o= /etc/dovecot/dovecot-sql.conf
chgrp vmail /etc/dovecot/dovecot.conf
chmod u=rw,g=r,o= /etc/dovecot/dovecot.conf
/etc/init.d/dovecot restart

To get some more logging in the case something goes wrong. Uncomment the folling and change to yes.

auth_verbose = yes
auth_debug = yes
auth_debug_passwords = yes

Dovecot SSL configuration

Out of the (Debian) box, Dovecot is SSL-enabled. We'll now replace the generic SSL-certificate, generated for the host by the Dovecot installation script, by our own certificate. To ths end, we first generate an appropriate certificate, e.g. an SSL wildcard certificate: "*.saruman.biz". This results in a public and a private certificate, both of which must be placed somewhere where Dovecot expects them and can read them.

By default, Dovecot expects the following locations/filenames/owner/attributes:

/etc/ssl/certs/dovecot.pem    -rw-r--r--  root:dovecot  4624 bytes
/etc/ssl/private/dovecot.pem  -rw-------  root:dovecot  1743 bytes

If we may make a suggestion: rename the generated certificates in both locations to dovecot.pem.bak, place your generated certificates in their place, and set the owner/permissions like displayed here.

However, if you've generated your own keys, you might have used a passphrase on the private key. You'll have to tell dovecot about it in its configuration file /etc/dovecot/dovecot.conf. To this end, edit the corresponding section by uncommenting the "ssl_" lines, and using your private key password (e.g. "zM7Ertq12") in the appropriate line:

ssl_cert_file = /etc/ssl/certs/dovecot.pem
ssl_key_file = /etc/ssl/private/dovecot.pem

# If key file is password protected, give the password here. Alternatively
# give it when starting dovecot with -p parameter.
ssl_key_password = zM7Ertq12

Naturally you are free to place the keys in a different location, and/or use a different name...

Finishing up

This is a good moment to test your configuration, if you haven't been able to test your work inbetween. If everything works as expected, you can add bells and whistles to your configuration.

smtpd TLS encryption

The first addition we'd like to present covers the way Postfix handles incoming connections. Since authentication works, we can have Postfix distrust every (unauthenticated) connection:

postconf -e mynetworks=

Furthermore, since a default SSL certificate is installed by the Debian Postfix installation routine, we can encrypt all our connections using TLS; we enforce this using

postconf -e smtpd_use_tls=yes
postconf -e smtpd_tls_auth_only=yes

Of course, you need to adjust the settings of your IMAP client so that it properly uses TLS and properly authenticates itself.

If TLS works, you'll probably want to change the certificate as you have for Dovecot in the previous section. This is again pretty easy. If you haven't yet, now is the time to creating that custom SSL certificate for our mail server - but you have to make sure that you DON'T use a password on the private key. Unlike Dovecot, Postfix cannot be told the password to the private key somewhere.

For starters, look at the TLS-section of your main.cf that you currently have. Chances are a big chunk of it looks like this:

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes

Now all you need to do is replace the name of the snakeoil-keys with those of appropriate ssl-certificates, the private key of which needs to be NOT passphrase-protected. For this you could copy the Dovecot certificates, if you strip that passphrase in the manner described in the SSL certificate section. The main.cf then would become:

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/postfix.pem
smtpd_tls_key_file=/etc/ssl/private/postfix.key
smtpd_use_tls=yes

Restart Postfix, and presto.

Other additions

If everything still works after these changes, then congratulations, you now have a powerful, flexible and pretty secure mail setup. Of course, there are always points for improvements. We'll cover these in separate pages. One page that we want to direct you to now, is

• Antispam measures for our Postfix mailserver