Building a secure email server is a cornerstone of any modern IT infrastructure. Whether you’re powering a small business or a large enterprise, the combination of Postfix for SMTP and Dovecot for IMAP/POP3 gives you a battle-tested, flexible stack that can handle everything from simple notifications to complex mail routing. In this guide we’ll walk you through every step—from preparing the server to hardening Postfix, configuring Dovecot, and performing final tests. We’ll also highlight the most common mistakes and share insider tips to keep your mail system resilient and compliant.
What You’ll Need
- A clean Ubuntu 22.04 LTS (or Debian 12) VPS with root access
- A registered domain name (e.g., example.com)
- Basic knowledge of the Linux command line and text editors (vim, nano)
- Administrative access to your DNS provider
- OpenSSL installed for generating certificates (or access to a CA)
- Optional: Let’s Encrypt for free SSL certificates
- Firewall (ufw or iptables) configured to allow ports 25, 587, 993, 995
Step 1: Prepare the Server
Start by updating your system and installing the necessary packages. A clean environment reduces the risk of configuration drift.
# apt update && apt upgrade -y
# apt install -y postfix dovecot-core dovecot-imapd dovecot-pop3d dovecot-lmtpd dovecot-mysql dovecot-sieve dovecot-managesieved dovecot-ldap dovecot-sqlite
When prompted during the Postfix installation, choose “Internet Site” and set your system mail name to your fully-qualified domain name (FQDN), e.g., mail.example.com.
Create a dedicated mail user and mailbox directory:
# groupadd mail
# useradd -g mail -d /var/mail -s /usr/sbin/nologin mail
# mkdir -p /var/mail/vhosts/example.com
# chown -R mail:mail /var/mail/vhosts
# chmod -R 700 /var/mail/vhosts
We’ll use virtual mailboxes, so the actual email addresses will be stored in a MySQL database. Make sure the database is up and running before moving forward.
Step 2: Configure DNS – MX, SPF, DKIM, DMARC
DNS is the first line of defense against spam and spoofing. Here’s what you need:
- MX Record: Point to your mail server. Example:
example.com. IN MX 10 mail.example.com. - SPF Record: Authorize your server to send mail. Example:
v=spf1 mx a ip4:YOUR_IP -all - DKIM: Use OpenDKIM or Dovecot’s built-in signing. We’ll use OpenDKIM. Install it:
# apt install -y opendkim opendkim-tools
Generate keys:
# mkdir /etc/opendkim/keys/example.com
# opendkim-genkey -t -s mail -d example.com -D /etc/opendkim/keys/example.com
# chown opendkim:opendkim /etc/opendkim/keys/example.com/*
Add the selector to your DNS TXT record: mail._domainkey.example.com. IN TXT "v=DKIM1; k=rsa; p=PUBLIC_KEY_HERE"
v=DMARC1; p=reject; rua=mailto:[email protected]Step 3: Install and Configure Postfix
Open the main Postfix configuration file:
# nano /etc/postfix/main.cf
Add or update the following directives:
# Basic Settings
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
mydestination = $myhostname, localhost.$mydomain, localhost
relay_domains = $myhostname
mynetworks = 127.0.0.0/8
inet_interfaces = all
inet_protocols = ipv4
# Virtual Mailboxes
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-domains-maps.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp
# TLS Settings
smtpd_use_tls = yes
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level = may
smtpd_tls_protocols = TLSv1.2 TLSv1.3
smtpd_tls_ciphers = high
smtpd_tls_exclude_ciphers = aNULL,MD5,3DES
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
# SASL Authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
# DKIM Signing
smtpd_milters = inet:127.0.0.1:8891
milter_default_action = accept
milter_protocol = 6
# Logging
maillog_file = /var/log/mail.log
Next, create the MySQL map files referenced above. Example for mysql-virtual-domains-maps.cf:
user = mailuser
password = mailpass
hosts = 127.0.0.1
dbname = maildb
query = SELECT 1 FROM domain WHERE name='%s'
Do the same for mailbox and alias maps, adjusting the SQL queries accordingly.
Step 4: Harden Postfix – TLS, SPF, SPF Checks, and Rate Limiting
Generate a strong self-signed certificate or obtain one from Let’s Encrypt. For Let’s Encrypt, install certbot:
# apt install -y certbot
# certbot certonly --standalone -d mail.example.com
Update the TLS paths in main.cf to point to the new cert and key. Then, enable SPF checks by installing the Postfix SPF policy service:
# apt install -y postfix-policyd-spf-python
Configure Postfix to use it:
# nano /etc/postfix/main.cf
# Add
policy_time_limit = 3600
smtpd_recipient_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination,
check_policy_service inet:127.0.0.1:10023
Rate limiting is essential to mitigate abuse. Add a simple restriction:
# nano /etc/postfix/main.cf
# Add
smtpd_sender_restrictions =
reject_unknown_sender_domain,
reject_non_fqdn_sender,
reject_invalid_hostname,
reject_unlisted_sender,
reject_unauth_destination
Step 5: Install and Configure Dovecot
Open the Dovecot configuration:
# nano /etc/dovecot/dovecot.conf
Add or update:
!include_try /usr/share/dovecot/protocols.d/*.protocol
protocols = imap pop3 lmtp
mail_location = maildir:/var/mail/vhosts/%d/%n
userdb {
driver = passwd
}
passdb {
driver = pam
}
# SASL Authentication for Postfix
auth_mechanisms = plain login
!
Configure SSL in /etc/dovecot/conf.d/10-ssl.conf:
ssl = required
ssl_cert = Replace the placeholder with the path to your SSL certificate and key.
Enable LMTP for Postfix to deliver mail:
# nano /etc/dovecot/conf.d/10-master.conf
service lmtp {
unix_listener private/dovecot-lmtp {
mode = 0600
user = postfix
group = postfix
}
}
Step 6: Set Up SSL/TLS for Dovecot
Generate or import your certificates:
# cp /etc/letsencrypt/live/mail.example.com/fullchain.pem /etc/dovecot/ssl/mail.example.com.crt
# cp /etc/letsencrypt/live/mail.example.com/privkey.pem /etc/dovecot/ssl/mail.example.com.key
Update /etc/dovecot/conf.d/10-ssl.conf accordingly:
ssl_cert = Restart services:
# systemctl restart postfix
# systemctl restart dovecot
Step 7: Final Testing and Maintenance
Test inbound and outbound mail using swaks:
# apt install -y swaks
# swaks --to [email protected] --from [email protected] --server mail.example.com --tls
Check logs for any errors:
# tail -f /var/log/mail.log
Set up automatic certificate renewal with cron if using Let’s Encrypt:
# crontab -e
0 3 * * * /usr/bin/certbot renew --quiet --post-hook "systemctl reload dovecot"
Common Mistakes to Avoid
1. Leaving the default Postfix configuration: The default is intentionally permissive. Always set mydestination and relay_domains correctly.
2. Using weak TLS ciphers: Disable weak protocols and ciphers to avoid downgrade attacks. The snippet above ensures only TLS 1.2/1.3 and strong ciphers are used.
3. Forgetting to secure the LMTP socket: The LMTP socket should be owned by the postfix user and have strict permissions.
4. Not configuring SPF/DKIM/DMARC: Without these, your mail will be flagged as spam. Verify each record with tools like dig and dkimvalidator.com.
5. Ignoring logs: Regularly inspect /var/log/mail.log and /var/log/dovecot.log for anomalies.
Tips and Tricks
• Use fail2ban to automatically block IPs that attempt to brute-force SMTP authentication.
• Consider using opendkim with opendmarc to enforce DKIM signing and DMARC reporting.
• For large mailboxes, use maildir format to avoid locking issues.
• Keep your server’s kernel and packages updated; security patches often fix CVEs that could be exploited via mail protocols.
Frequently Asked Questions
Can I use Postfix and Dovecot on a shared hosting environment?
Yes, but you’ll need root access or the ability to configure the server. Shared hosts often restrict ports 25/587, so you may need to use an external relay or a VPS.
How do I handle spam filtering?
Integrate SpamAssassin or Rspamd. Configure Postfix to pass mail through the filter using content_filter and set up Dovecot to store flagged messages in a separate folder.
What is the best way to backup the mail server?
Use rsync or tar to snapshot /var/mail/vhosts and the MySQL database. Store backups in an offsite location and verify restore procedures regularly.
Conclusion
Setting up a secure email server with Postfix and Dovecot is a rewarding project that reinforces your understanding of mail protocols, security best practices, and system administration. By following this guide, you’ll have a robust, scalable solution that protects against spam, spoofing, and unauthorized access. Remember to keep your software up to date, monitor logs, and regularly audit your configuration. Happy mailing!
Photo by Stephen Phillips – Hostreviews.co.uk on Unsplash
