Cyrus: Build your own email server

Don’t trust Google? We’ll help you navigate the sea of acronyms to build your own mailserver.

You can’t beat the convenience and ease of use offered by Gmail. But unfortunately, all that free storage comes at a price: your privacy. Spam, intrusive adverts and snooping from unnamed government agencies are the inevitable downside of using someone else’s service for free. So why not build your own email server including anti-spam, anti-virus and webmail?

You can use your own server to retrieve messages from other mailservers, such as those provided by internet service providers, or other services like those from Google and Yahoo. But you don’t need to rely on others if you have your own server. If you have a domain name that you control, and if you can give your server a static public IP address then you can receive email directly.

We’re going to implement a sealed server, which means that users cannot log in to it. They have email accounts that are only accessible using client applications that connect to the server using IMAP, the Internet Message Access Protocol (we could, but won’t, also use the older Post Office Protocol, POP).

At the heart of the system is the IMAP server, Cyrus. This accepts messages using a protocol called the Local Mail Transfer Protocol, or LMTP, and stores them in mailboxes – it’s a mail delivery agent. Users can access their mail by connecting to the server using any IMAP-capable email client application.

You will need a, preferably new, server for this project and you’ll need root access to it. Our examples use Arch Linux, and we created a new virtual server.

Begin by installing Cyrus (build the Arch User Repository package first – see the boxout below):

$ pacman -U ~build/cyrus-imapd/cyrus-imapd-2.4.17-5-x86_64.pkg.tar.xz

The default configuration writes data to /var/imap and user mailboxes to /var/spool/imap. You can change this if you prefer another location; we’ll configure our server to use /srv/mail/cyrus to illustrate this. If you follow suit, you can also delete the default locations:

rm -r /var/spool/imap /var/imap

Some command line tools are installed to /usr/lib/cyrus/bin so it’s worth extending your PATH (do it in
/etc/profile to make this permanent):

export PATH=”$PATH”:/usr/lib/cyrus/bin

There are two configuration files, and the first of these is /etc/cyrus/cyrus.conf. It defines the services that the server will offer, and the default file is generally acceptable unless, like us, you want to change the data path. This requires one entry in the file to be altered:

lmtpunix cmd=”lmtpd” listen=”/srv/mail/cyrus/socket/lmtp” prefork=0

The listen argument points to the Unix domain socket where the server accepts LMTP protocol connections. We change this to be in a subdirectory of our chosen data path. You can also take this opportunity to disable unwanted services; we commented out pop3 and pop3s because we plan to offer IMAP-only access.

The second file, /etc/cyrus/imapd.conf, configures the IMAP server and needs to be written from scratch. The following example will get you started, but you may want to read the documentation and configure it to meet your needs.

configdirectory: /srv/mail/cyrus
partition-default: /srv/mail/cyrus/mail
admins: cyrus
sasl_pwcheck_method: saslauthd
sasl_saslauthd_path: /var/run/saslauthd/mux
sasl_mech_list: PLAIN
allowplaintext: yes
altnamespace: yes
unixhierarchysep: yes
virtdomains: userid
defaultdomain: mydomain.com
hashimapspool: true
sieve_admins: cyrus
sievedir: /srv/mail/cyrus/sieve

This tells Cyrus to use /srv/mail/cyrus for its configuration and, within that, a mail subdirectory where it should store mail. Virtual domains allows domain-specific mailboxes – you can have accounts for alice@example-one.com and alice@example-two.com. The defaultdomain is the domain that unqualified user accounts, like “alice”, belong to.

To improve the end-user experience, we set altnamespace so that users’ email folders appear alongside, rather than within, their inbox, and unixhierarchysep delimits mail folders with slashes instead of the default, which is to use a period.

You can give your test account a meaningful name and enter your own name in the identity section.

SASL

Our configuration uses SASL for authentication. This is the Simple Authentication and Security Layer, and was automatically installed as a dependency of the IMAP server. We just use the default configuration here, which passes plain-text passwords to the saslauthd daemon that, in the default configuration on Arch Linux, uses PAM for authentication. This is acceptable for a test system, but you should consider configuring SASL to use more secure methods that satisfy your own security requirements.

So, create a test account for testing and verify that SASL can authenticate it. The default SASL configuration authenticates system users so we use a nobody account that can be authenticated but cannot be used to log in to the server.

$ useradd -c ‘Test email account’ -u 99 -o -g nobody -d /dev/null -s /bin/false testuser
$ echo testuser:testpass | chpasswd

Start saslauthd (also enable it so that it starts on boot) and test that SASL authentication works for the new test user:

$ systemctl enable saslauthd
$ systemctl start saslauthd
$ testsaslauthd -u testuser -p testpass
0: OK “Success.”

The installation also created a cyrus user, and the server’s processes run as this user. We can also use it for administrative tasks if we set its home directory, shell and password:

$ usermod -s /bin/bash -d /srv/mail/cyrus cyrus
$ echo cyrus:cyrus | chpasswd

To complete the configuration, make the required directories and build the IMAP folders:

$ mkdir -p -m 750 /srv/mail/cyrus/mail
$ chown -R cyrus:mail /srv/mail/cyrus
$ su cyrus -c ‘mkimap /etc/cyrus/imapd.conf’

Now start the server

$ systemctl enable cyrus-master
$ systemctl start cyrus-master

Test IMAP access for the test user

$ telnet localhost imap
. login testuser testpass
. logout

If everything went well, the server responses will begin with * OK. You can now set up your email client to connect to the IMAP account, but it doesn’t have any folders yet. The cyradm tool is used to create mailboxes, and the minimum is an inbox:

$ su cyrus -c ‘cyradm -u cyrus -w cyrus localhost
localhost.localdomain> cm user/testuser

You can then use your email client to create subfolders, or you can use cyradm – cm creates mailboxes (folders) and lm lists them:

localhost.localdomain> cm user/testuser/Sent
localhost.localdomain> lm
user/testuser (HasChildren)
user/testuser/Sent (HasNoChildren)
user/testuser/Trash (HasNoChildren)

You can now send a message to the test user. Create a test message in a file (call it testmessage) with the following contents (the empty line is required – it marks the beginning of the message body).

From: Test Message <test@example.com>
Subject: This is a test message
This is a basic test e-mail message

To send the message into Cyrus, use the deliver tool and then check your email client for the message.

deliver testuser < testmessage

That completes the configuration of the IMAP server. It’s ready to receive mail and can serve it to users’ email clients, but nothing is yet being sent to it.

You can specify the server by its host name or IP address. The username is the IMAP “testuser” account that we set up on the server.

The simplest way to get mail into your server is to fetch it from another one. A daemon known as a Mail Retrieval Agent (MRA) can fetch mail from remote IMAP or POP mailboxes such as your Gmail account. The MRA that we’ll use is called Fetchmail:

$ pacman -S fetchmail

Fetchmail takes instructions from /etc/fetchmailrc, which must be set with 0700 permissions. The file begins with global settings and defaults and it’s here that we tell Fetchmail to deliver all mail to our server’s LMTP socket.

defaults
smtphost “/srv/mail/cyrus/socket/lmtp”
smtpaddress mydomain.com

Specify the same domain here as the defaultdomain in /etc/cyrus/imapd.conf. Without this, any unqualified usernames will have localhost appended and the mailserver won’t recognise them.

With the defaults configured, what remains is to provide blocks for each remote server that we wish to fetch from. You can fetch messages from many remote accounts and deliver them to any configured local email account. Here is an example that fetches from Gmail:

poll poll imap.gmail.com protocol imap
user alice@gmail.com there pass abc123 is alice here
user alice_other@gmail.com there pass secretword is alice here
user jane.doe@gmail.com there pass secretword is jane here

and similar examples for Yahoo and Microsoft mail accounts:

poll pop.mail.yahoo.com protocol pop3
user johndoe there pass mypassword is john here ssl
poll pop3.live.com protocol pop3
user bob@hotmail.com there pass 123abc is bob here ssl

You can fetch mail on demand (the optional -v makes it verbose):

$ fetchmail -v -f /etc/fetchmailrc

Or, what you will most likely want to do is start it as a daemon that regularly polls for available messages. The daemon on Arch Linux runs as the fetchmail user and requires that it owns the /etc/fetchmail file. We can start the daemon:

$ chown fetchmail /etc/fetchmailrc
$ systemctl enable fetchmail
$ systemctl start fetchmail

Fetchmail will poll at an interval defined by its systemd unit. On Arch Linux this is 900 seconds (15 minutes). You can use the SIGHUP signal to instruct the daemon to poll on demand.

$ pkill -USR1 fetchmail

We now have a working email server that fetches email from other external mailservers. We can improve upon that by having mail sent to us.

MXToolbox.com can test your server from outside…

Join the Postal Union

Email is sent across the internet by Mail Transfer Agents. These aren’t trench-coated sleuths but network services that converse using the Simple Mail Transfer Protocol, or SMTP. We need to join in this conversation so that we can receive email – we need our own Mail Transfer Agent, and we’ll use Postfix; it’s a straightforward installation from the repository:

$ pacman -S postfix

Postfix is controlled by a configuration file called main.cf, and you’ll find it in /etc/postfix. It contains a large number of options but most of the defaults are acceptable for our needs.

Our mailserver supports mail accounts for multiple domains, so we’ll configure Postfix to recognise these Virtual Mailbox Domains and deliver any mail received for them into our mailserver’s LMTP interface.

virtual_mailbox_domains = mydomain.com myotherdomain.co.uk
virtual_transport = lmtp:unix:/srv/mail/cyrus/socket/lmtp

Start the Postfix server and tail its journal so that you can see what it does:

$ systemctl enable postfix
$ systemctl start postfix
$ journalctl -f -u postfix &

You can use Telnet to send a test message. You should be able to see it in your email client as soon as you’ve sent it.

$ telnet localhost smtp
EHLO example.com
MAIL FROM:bob@example.com
RCPT TO:testuser@mydomain.com
DATA
From: Bob <bob@example.com>
Subject: This is a test message
This is a test SMTP message
.
QUIT

The test confirms that our server can deliver emails received for our domains over SMTP but, before anything can be sent to it, it needs a static public IP address and the domains’ DNS records need to be updated with that address so that other Mail Transfer Agents can find it.

Speak to me

Your internet service provider allocates you a public IP address for your connection. You will need to ensure this is static. If in any doubt, contact your ISP. We’ll use the public address of example.com in our examples, which is 93.184.216.119.

You’ll need to open the SMTP port (25) on your perimeter firewall and configure a NAT translation to connect that port to your mailserver. How you do this will depend on what networking hardware you have. The following examples assume that 93.184.216.119:25 reaches your Postfix SMTP interface. Once you have a static IP address that connects to your server, you should configure your domains’ DNS records. How you do this depends on the tools provided by your DNS provider, usually the registrar of your domains.

You need to configure two records: an address record (A record) that points to your static public IP address, and a mail exchange record (MX record) that points to the A record. DNS records have four fields but each record only uses three of them. Configure the A record like this:

Left field: mail
Type: A
Priority: <blank>
Right field: 93.184.216.119

and the MX record like this:

Left field: <blank>
Type: MX
Priority: 5
Right field: mail

The MX record references the A record by name (we imaginatively chose to call ours “mail”). The A record gives the IP address of the server. Both records are required – the MX record cannot contain an IP address. Remember that DNS updates can take up to 48 hours to take effect.

You can define multiple MX records and use the priority field to order them. If you do this then delivery is attempted using each MX record in ascending priority order until one succeeds. If delivery fails then the message is returned to the sender (it’s bounced). You could use multiple MX records to have mail delivered to a mailbox at your ISP if your own server is offline. Your server’s Mail Retrieval Agent, Fetchmail, could then retrieve any such mail when it comes back online.

You can perform various tests to ensure that your server can accept mail. You can probe your port (https://www.grc.com/x/portprobe=25) and test your MX records, either online with http://mxtoolbox.com or on the command line with dig:

$ dig +short MX mydomain.com
5 mail.mydomain.com.
$ dig +short A mail.mydomain.com
93.184.216.119

Now that your SMTP server is on the internet you need to make sure it’s properly configured, otherwise it won’t be long before spammers find it and start using it to distribute their wares. You can use http://mxtoolbox.com/SuperTool.aspx to check how your server responds to the outside world and confirm that you aren’t offering an open relay to spammers; https://www.wormly.com/pervertedness lets you send test emails into your server.

We’ve configured enough to receive, store and serve email to multiple users over IMAP. Next time, we’ll start filtering out unwanted messages, like anything containing spam or viruses or even just mails from people we just don’t like. We’ll also let our users send email, because it’s good to talk.