In the first part of this article we talked about the theory, how Postfix and Dovecot work. If you've missed it, I strongly recommend you to read it to understand the basics. Otherwise, if something won't work, you'll don't know how to fix it.
As I said in the first part of this article, I want to describe here, how I've set up my own mail server using Postfix (MTA), Dovecot (MDA/MRA), Rspamd (anti-spam protection), and set up SPF/DKIM/DMARC. As a result, you'll have your own mail server where you can add an unlimited number of domains and mailboxes.
I hope this article will help you to set everything up, but configs are strongly depend on how you want your mail server works. I need a mail server to manage all of the mailboxes in different domains of my own projects (support@domain.com, etc.). If you want to make some changes in configuration, read the official documentations, or the first part of this article.
Database schema
I'll store all of my domains and mailboxes in PostgreSQL to have a single source of data for Postfix and Dovecot. They will connect to the database directly.
It's also possible to generate an lmdb lookup table (or other type) for Postfix (or use dynamic recipient verification) and passwd-file for Dovecot by means of a cron script. The authentication process will be faster in both Postfix and Dovecot, but changes in the database will be not available until the cron script runs the next time.
By the way, dynamic recipient verification allows Postfix to make a request to Dovecot using the same LMTP daemon to check whether the user exists (Postfix trying to send a test email, but send the QUIT command after RCPT TO). So Postfix can reject emails for non-existent users without having access to the database or having a separate lookup table for this purpose. It's useful when Postfix shouldn't have access to the database, or when you store list of users in the file /etc/dovecot/users which cannot be read by Postfix. Postfix also caches all requests to make further checks much faster and stores them in the file specified in address_verify_map. By default, non-existent addresses expire after 3 days, positive addresses 31 days. See parameters starting with address_verify_. Use reject_unverified_recipient in smtpd_recipient_restrictions after permit_mynetworks, so Postfix will start making dynamic recipient verifications.
Here is the PostgreSQL schema:
CREATE DATABASE mail;
-- \c mail
CREATE TABLE domains (
id serial PRIMARY KEY,
created_at timestamptz DEFAULT now() NOT NULL,
name varchar(255) NOT NULL,
UNIQUE (name)
);
CREATE TABLE mailboxes (
id serial PRIMARY KEY,
created_at timestamptz DEFAULT now() NOT NULL,
domain_id int REFERENCES domains(id) ON DELETE CASCADE NOT NULL,
name varchar(64) NOT NULL,
password_hash text NOT NULL,
UNIQUE (domain_id, name)
);
-- Change the passwords here and in the following files:
-- /etc/postfix/relay_domains.cf
-- /etc/postfix/relay_recipients.cf
-- /etc/dovecot/dovecot-sql.conf.ext
CREATE USER postfix WITH PASSWORD 'postfix';
CREATE USER dovecot WITH PASSWORD 'dovecot';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO postfix;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO dovecot;Limits have not been chosen spontaneously. According to RFC 3696, the maximum length of the local part of email addresses must be 64 characters, and the domain part – 255 characters.
Postfix
Postfix will be just a front relay and hand off emails to Dovecot. So I'll use the local address class only for root@localhost (used for important system alerts) and the relay address class for domains stored in the database.
apk add postfix postfix-pgsql/etc/postfix/main.cf:
# --------------------------------------------------
# Defaults
# --------------------------------------------------
# Compatibility
compatibility_level = 3.8
# Local pathname information
queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
# Queue and process ownership
mail_owner = postfix
# Debugging control
debug_peer_level = 2
debugger_command =
PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
ddd $daemon_directory/$process_name $process_id & sleep 5
# Install-time configuration information
sendmail_path = /usr/sbin/sendmail
newaliases_path = /usr/bin/newaliases
mailq_path = /usr/bin/mailq
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
sample_directory = /etc/postfix
readme_directory = /usr/share/doc/postfix/readme
inet_protocols = ipv4
meta_directory = /etc/postfix
shlib_directory = /usr/lib/postfix
# --------------------------------------------------
# Must be a fully qualified domain
myhostname = mail.domain.com
mydomain = domain.com
# This domain will be appended to addresses without a domain
myorigin = $mydomain
# Use only ipv4 when making or accepting connections.
inet_protocols = ipv4
# The directory where unix-style mailboxes are kept.
# A trailing slash indicates that the Maildir storage format should be used.
mail_spool_directory = /var/mail/
# Local address class
local_recipient_maps = lmdb:/etc/postfix/local_recipients
# Relay address class
relay_domains = pgsql:/etc/postfix/relay_domains.cf
relay_recipient_maps = pgsql:/etc/postfix/relay_recipients.cf
transport_maps = $relay_domains
# What other systems can use this mail server to send emails.
# If `mynetworks` is set, `mynetworks_style` is ignored.
mynetworks_style = host
mynetworks =
# Select logging to a file
maillog_file = /var/log/postfix.log
maillog_file_permissions = 0644
# Restrict mail delivery to external commands and files
allow_mail_to_commands =
allow_mail_to_files =
# Max number of recipients for a single email
smtpd_recipient_limit = 10
# Max size of an email Postfix will accept
message_size_limit = 10240000
# Restrictions to prevent spam.
# Use very carefully:
# `reject_unknown_client_hostname` because many legitimate domains don't have the PTR record.
# `reject_non_fqdn_helo_hostname` and `reject_unknown_helo_hostname` because many clients don't use a fully qualified hostname.
# Use `warn_if_reject <restriction>` for testing.
disable_vrfy_command = yes
smtpd_helo_required = yes
# Required for `reject_sender_login_mismatch` (used by the submission agent)
smtpd_sender_login_maps = pgsql:/etc/postfix/relay_recipients.cf
smtpd_client_restrictions =
smtpd_helo_restrictions =
reject_invalid_helo_hostname
smtpd_sender_restrictions =
reject_non_fqdn_sender
reject_unknown_sender_domain
smtpd_recipient_restrictions =
reject_non_fqdn_recipient
reject_unknown_recipient_domain
permit_mynetworks
reject_unauth_destination
# Check quota status of the user on the IMAP server
check_policy_service unix:/var/run/dovecot/quota-status
smtpd_data_restrictions =
reject_unauth_pipelining
# SASL authentication.
# It's activated and used only by the submission agent (see master.cf).
# It's not used by other MTAs.
smtpd_sasl_auth_enable = no
smtpd_sasl_type = dovecot
smtpd_sasl_path = /var/run/dovecot/auth
# Make sure plaintext mechanisms are used only with SSL/TLS
smtpd_sasl_security_options = noanonymous, noplaintext
smtpd_sasl_tls_security_options = noanonymous
# TLS
smtpd_tls_security_level = may
# Use `may` for better compatibility with MTAs on the Internet (unfortunately)
smtp_tls_security_level = may
smtpd_tls_key_file = /etc/ssl/private/mail.domain.com.key
smtpd_tls_cert_file = /etc/ssl/certs/mail.domain.com.crt
smtpd_tls_dh1024_param_file = /etc/ssl/private/mail.domain.com.dh
# DKIM, Rspamd
smtpd_milters =
unix:/var/run/opendkim/opendkim.sock
unix:/var/run/rspamd/rspamd-proxy.sock
non_smtpd_milters = $smtpd_milters
milter_default_action = accept/etc/postfix/local_recipients:
root -You can also create your own mailbox for domain.com using the database (see above) and create an alias in /etc/postfix/aliases like this root: name@domain.com so all system emails will be redirected to name@domain.com.
/etc/postfix/relay_domains.cf:
hosts = localhost:5432
dbname = mail
user = postfix
password = postfix
query =
SELECT 'lmtp:unix:/var/run/dovecot/lmtp'
FROM domains WHERE name = '%s'If you want to connect to your database via unix socket, specify hosts like this hosts = unix:/var/run/postgresql.
Use lmtp:unix:private/lmtp-dovecot if Postfix is running in a chroot environment to open an additional LMTP socket in the /var/spool/postfix/private/lmtp-dovecot (see below).
/etc/postfix/relay_recipients.cf:
hosts = localhost:5432
dbname = mail
user = postfix
password = postfix
query =
SELECT '%s' FROM mailboxes
INNER JOIN domains ON
domains.id = mailboxes.domain_id AND
domains.name = '%d'
WHERE mailboxes.name = '%u'relay_recipients.cf is used in 2 places: relay_recipient_maps and smtpd_sender_login_maps. The result from the lookup table specified in relay_recipient_maps is not used and can be any (for example, we can return SELECT 1), but the lookup table specified in smtpd_sender_login_maps must return a list of SASL login names separated by comma and/or whitespace. It doesn't make sense to create different lookup table for smtpd_sender_login_maps, so we can just return SELECT '%s' instead of SELECT 1 and use it in both places.
Add the submission agent in the master.cf file:
submission inet n - n - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_tls_auth_only=yes
-o local_header_rewrite_clients=static:all
-o smtpd_reject_unlisted_recipient=no
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=reject_sender_login_mismatch
-o smtpd_relay_restrictions=
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
-o milter_macro_daemon_name=ORIGINATINGDovecot
apk add dovecot dovecot-pgsql dovecot-lmtpdCreate the same UID and GID for all the users.
addgroup -g 10000 virtual
adduser -u 10000 -G virtual -s /sbin/nologin -D virtualCreate a directory for storing email data.
mkdir /srv/vmail
chown virtual:virtual /srv/vmail
chmod 770 /srv/vmailGenerate a self-signed key pair (replace the domain).
openssl req -new -newkey rsa:2048 -days 3650 \
-nodes -x509 -subj /CN=mail.domain.com \
-keyout /etc/ssl/private/mail.domain.com.key \
-out /etc/ssl/certs/mail.domain.com.crt
chmod 400 /etc/ssl/private/mail.domain.com.key
chmod 444 /etc/ssl/certs/mail.domain.com.crtGenerate Diffie-Hellman parameters.
openssl dhparam -out /etc/ssl/private/mail.domain.com.dh 2048
chmod 400 /etc/ssl/private/mail.domain.com.dhThe recommended number of bits is 2048.
If you want to run a mail server in a docker container, like me, generate and store all certificates on the host machine.
# List of used plugins
mail_plugins = $mail_plugins zlib quota
# List of supported protocols
protocols = lmtp imap
# Transmit a password in plain text
auth_mechanisms = plain login
# Disable plaintext mechanisms unless a SSL/TLS or local connection is used
disable_plaintext_auth = yes
# List of IPs or network ranges that considered secure even without a SSL/TLS connection.
# Use it for special clients such as webmailers, internal monitoring, Dovecot proxy servers.
login_trusted_networks =
# SSL/TLS is required for all client connections
ssl = required
ssl_cert = </etc/ssl/certs/mail.domain.com.crt
ssl_key = </etc/ssl/private/mail.domain.com.key
ssl_dh = </etc/ssl/private/mail.domain.com.dh
# Prefer server ciphers and their order over client's list.
ssl_prefer_server_ciphers = yes
# To transfer the entire lowercased email address to the authentication process
auth_username_format = %Lu
# Master users.
# They are able to log in under of any user, but not as themselves (name@domain.com*master).
# It's better to use it only temporarily for debugging or migration purposes.
# auth_master_user_separator = *
# passdb {
# driver = passwd-file
# args = /etc/dovecot/master-users
# master = yes
# # Verify whether a user exists before allowing the master user to log in.
# # Otherwise, a new user can be created.
# result_success = continue
# }
# Regular users
passdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
# Must be placed before the actual userdb lookup so that
# Dovecot can cancel if it already has all the required values.
userdb {
driver = prefetch
}
userdb {
driver = sql
args = /etc/dovecot/dovecot-sql.conf.ext
}
mail_location = maildir:%h/Maildir
namespace inbox {
# Hierarchy separator (use "/", not ".").
# If you'll use the "." separator, and a username will contain a point,
# the shared folders might be displayed incorrectly.
separator = /
# Prefix required to access this namespace (use "INBOX/", not "INBOX").
# If you'll use the "INBOX" prefix, a problem will be occured when a user
# sets up a regular folder named "shared" (as well as the "public" folder).
prefix = INBOX/
type = private
hidden = no
ignore_on_failure = no
inbox = yes
list = yes
location =
subscriptions = yes
mailbox Archive {
auto = subscribe
special_use = \Archive
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
autoexpunge = 30d
autoexpunge_max_mails = 100000
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
autoexpunge = 30d
autoexpunge_max_mails = 100000
}
}
# These settings are recommended with `autoexpunge`.
mailbox_list_index = yes
mail_always_cache_fields = date.save
# Activate the LMTP daemon.
service lmtp {
# Open a unix socket in `/var/run/dovecot/lmtp`.
unix_listener lmtp {
}
# If Postfix is running in a chroot environment, open an additional socket.
# unix_listener /var/spool/postfix/private/lmtp-dovecot {
# user = postfix
# group = postfix
# }
}
# Provide a SASL interface for Postfix via a unix socket. Postfix will pass
# authentication queries to Dovecot via that interface and receives a yes/no response.
# SASL is used by the submission agent in Postfix.
service auth {
# Open a unix socket in `/var/run/dovecot/auth-userdb`.
# Dovecot uses this unix socket for its own purposes. Don't use it in Postfix.
unix_listener auth-userdb {
}
# Open a unix socket in `/var/run/dovecot/auth`.
# Used by Postfix.
unix_listener auth {
user = postfix
group = postfix
mode = 0666
}
# If Postfix is running in a chroot environment.
# unix_listener /var/spool/postfix/private/auth {
# user = postfix
# group = postfix
# mode = 0666
# }
}
protocol lmtp {
# Enable sieves to move emails marked as spam to the junk IMAP folder.
mail_plugins = $mail_plugins sieve
}
protocol imap {
# 1. Enable the client and server agree on an on-the-fly compression of
# the IMAP transmission in order to minimize the required bandwidth.
# 2. Enable quota rules.
# 3. Enable sieves to train Rspamd when a user moves an email to/from
# the junk IMAP folder.
mail_plugins = $mail_plugins imap_zlib imap_quota imap_sieve
}
# Save and read all emails in a compressed state via a zlib library.
# It reduces space consumption by around 40%, relieves the hard drive from
# the strain of read operations, backup process runs faster, but it slightly
# increases the CPU usage.
# Make sure you've added `zlib` in `mail_plugins`.
plugin {
zlib_save = gz
zlib_save_level = 6
}
# Quota.
plugin {
# Quotas for the individual mailbox of every user.
quota = maildir:User quota
quota_rule = *:storage=1G
quota_rule1 = INBOX/Trash:storage=+10%%
quota_rule2 = INBOX/Spam:storage=+20%%
quota_rule3 = INBOX/Sent:ignore
# Allow users to exceed their quota limit when saving the latest email
quota_grace = 10%%
quota_warning = storage=100%% quota-warning 100 %u
quota_warning2 = storage=95%% quota-warning 95 %u
quota_warning3 = storage=80%% quota-warning 80 %u
quota_warning4 = -storage=100%% quota-warning -100 %u
# Quotas for all mailboxes in one domain.
# quota2 = dict:Domain quota:%d:file:/srv/vmail/dovecot-domain-quota
# quota2_rule = *:storage=10G
# quota2_rule1 = INBOX/Sent:ignore
# quota2_warning = storage=100%% domain-quota-warning 100 %u
# quota2_warning2 = storage=95%% domain-quota-warning 95 %u
# quota2_warning3 = storage=80%% domain-quota-warning 80 %u
# quota2_warning4 = -storage=100%% domain-quota-warning -100 %u
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
user = virtual
unix_listener quota-warning {
user = virtual
group = virtual
}
}
# The quota policy server for Postfix.
# It allows Postfix immediately reject an email if the quota has been exceeded.
service quota-status {
executable = quota-status -p postfix
# Open a unix socket in `/var/run/dovecot/quota-status`.
unix_listener quota-status {
user = virtual
group = virtual
mode = 0666
}
client_limit = 1
}
# Which feedback is returned to Postfix
plugin {
quota_status_success = DUNNO
quota_status_nouser = DUNNO
quota_status_overquota = "552 5.2.2 Mailbox is full"
}
# Train Rspamd when a user moves an email from/to the junk IMAP folder.
# Do not forget to add the sieve plugin for lmtp and imap protocols.
plugin {
sieve = file:%h/sieve;active=/home/virtual/sieve/active.sieve
}
plugin {
sieve_plugins = sieve_imapsieve sieve_extprograms
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
# The directory contains the scripts that are available for the pipe command.
sieve_pipe_bin_dir = /home/virtual/sieve
# When a user moves an email to Junk.
imapsieve_mailbox1_name = INBOX/Junk
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_before = file:/home/virtual/sieve/learn-spam.sieve
# When a user moves an email from Junk.
imapsieve_mailbox2_from = INBOX/Junk
imapsieve_mailbox2_name = *
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_before = file:/home/virtual/sieve/learn-ham.sieve
}
# The log file to use for error messages.
log_path = /var/log/dovecot.log
# Shows a username, IP address, SSL/TLS status,
# executed IMAP command of the connected client.
verbose_proctitle = yes
# Logging detailed messages on the authentication process
# auth_verbose = yes
# Logging authentication prompts
# auth_debug = yes
# Logging how dovecot is looking for the user's mail directory, how it analyzes it, etc.
# mail_debug = yes/etc/dovecot/master-users:
master:{PLAIN}secret::::::Change the name master and password secret of your master user, run dovecot reload, and check your master user by running doveadm auth test name@domain.com*master. From now on you can log in under any user like this name@domain.com*master.
/etc/dovecot/dovecot-sql.conf.ext:
driver = pgsql
connect = host=localhost port=5432 dbname=mail user=dovecot password=dovecot
default_pass_scheme = ARGON2ID
password_query = \
SELECT \
mailboxes.name username, \
domains.name domain, \
mailboxes.password_hash password, \
'/srv/vmail/%Ld/%Ln' userdb_home, \
10000 userdb_uid, \
10000 userdb_gid \
FROM mailboxes \
INNER JOIN domains ON \
domains.id = mailboxes.domain_id AND \
domains.name = '%Ld' \
WHERE mailboxes.name = '%Ln'
user_query = \
SELECT \
'/srv/vmail/%Ld/%Ln' home, \
10000 uid, \
10000 gid \
FROM mailboxes \
INNER JOIN domains ON \
domains.id = mailboxes.domain_id AND \
domains.name = '%Ld' \
WHERE mailboxes.name = '%Ln'
iterate_query = \
SELECT \
mailboxes.name username, \
domains.name domain \
FROM mailboxes \
INNER JOIN domains ON domains.id = mailboxes.domain_idIf you want to connect to your database via unix socket, specify connect like this connect = host=/var/run/postgresql dbname=mail user=dovecot password=dovecot.
Notes:
- %Ld is a lowercased domain part, and %Ln is a lowercased local part of email address.
- Do not remove user_query because in spite of password_query supports for prefetching, the user_query is still used when an email is received or the shared folder is being accessed.
- If a user name changes, you also have to rename the user's home directory on time. To avoid such a problem, you can store the home directory in the database.
- It's recommended to return uid, gid, home in the SQL queries instead of using mail_uid, mail_gid, and variables %u, %n, %d in mail_location.
- iterate_query is used by the doveadm purge command or when you run doveadm user '*' to get a list of all of the users in your script.
- You can specify multiple hosts in order to achieve load distribution and fail safety.
/usr/local/bin/quota-warning.sh:
#!/bin/sh
PERCENT=$1
USER=$2
STATUS=""
if [[ $PERCENT = "-100" ]]; then
STATUS="not overfull"
else
STATUS="$PERCENT% full"
fi
cat << EOF | /usr/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing"
From: postmaster@domain.com
Subject: Quota warning
Your mailbox is now $STATUS.
EOFnoenforcing is used to deliver quota warnings even if a mailbox is completely full.
chmod 710 /usr/local/bin/quota-warning.shDKIM
apk add opendkim opendkim-utils
mkdir /run/opendkim
chown opendkim:mail /run/opendkim
chown -R opendkim:mail /etc/opendkim/etc/opendkim/opendkim.conf:
BaseDirectory /run/opendkim
# Automatically re-start on failures.
AutoRestart yes
# The maximum automatic restart rate (limits the restarts to 10 in one hour).
AutoRestartRate 10/1h
# Select the canonicalization method(s) to be used when signing messages.
# Allows some reformatting of the header but not in the message body.
Canonicalization relaxed/simple
# Which mode(s) of operation are desired (s - signer, v - verifier).
Mode sv
# The signing algorithm used when generating signatures.
SignatureAlgorithm rsa-sha256
# Attempts to become the specified userid before starting operations.
UserID opendkim
# The socket that should be established by the filter to receive connections from sendmail.
# It opens a socket `/run/opendkim/opendkim.sock`. If Postfix is running in
# a chroot environment, change it to `/var/spool/postfix/private/opendkim`.
Socket local:opendkim.sock
# Permissions mask to be used for file creation.
UMask 002
# The location of a file mapping key names to signing keys.
KeyTable refile:/etc/opendkim/key_table
# A table used to select one or more signatures to apply to a message based on
# the address found in the From: header field.
SigningTable refile:/etc/opendkim/signing_table
# A set of "external" hosts that may send mail through the server as one of
# the signing domains without credentials as such.
ExternalIgnoreList refile:/etc/opendkim/trusted_hosts
# A set internal hosts whose mail should be signed rather than verified.
InternalHosts refile:/etc/opendkim/trusted_hosts
# Enable logging.
Syslog yes
# Enable logging about successful signing or verification of messages.
SyslogSuccess yes
# Enable logging about the logic behind the filter's decision to either sign
# a message or verify it.
LogWhy yes/etc/opendkim/gen-dkim-key.sh:
#!/bin/sh
DOMAIN=$1
DIR="/etc/opendkim/keys/$DOMAIN"
SELECTOR="mail"
KEY_TABLE="/etc/opendkim/key_table"
SIGNING_TABLE="/etc/opendkim/signing_table"
TRUSTED_HOSTS="/etc/opendkim/trusted_hosts"
if [ ! -f "$TRUSTED_HOSTS" ]; then
cat > "$TRUSTED_HOSTS" <<- EOF
127.0.0.1
::1
localhost
EOF
fi
mkdir -p "$DIR"
opendkim-genkey -b 1024 -d "$DOMAIN" -s "$SELECTOR" -D "$DIR" -r
echo "$SELECTOR._domainkey.$DOMAIN $DOMAIN:$SELECTOR:$DIR/$SELECTOR.private" >> "$KEY_TABLE"
echo "*@$DOMAIN $SELECTOR._domainkey.$DOMAIN" >> "$SIGNING_TABLE"
echo "$DOMAIN" >> "$TRUSTED_HOSTS"
cat "$DIR/$SELECTOR.txt"Rspamd
Rspamd is high-performance spam filtering system. It can communicate with Postfix using the milter protocol.
Do you remember how Postfix receives emails via SMTP? The smtpd daemon receives an email, sends it to the cleanup daemon, which deposits it in the incoming queue, and notifies the queue manager qmgr (see more in the first part of this article). Before sending an email to the cleanup daemon, the smtpd daemon can hand off the email to a milter, which can make some changes with it and then hand it off back to the smtpd daemon. For example, a milter can determine whether the email is spam, and if so, it adds a new header X-Spam: true to the email. The same way the opendkim milter works. You send an email to Postfix, the smtpd daemon receives it, hands it off to the opendkim milter, which adds the DKIM-Signature to the email, and hand it off back to the smtpd daemon.
Rspamd has 4 workers:
- Proxy worker interacts with an MTA using the milter protocol and forwards emails to a normal worker. It can also work in the self scan mode and do everything by itself without normal workers.
- Normal worker scans emails for spam. It communicates with a fuzzy storage worker to get emails hashes to determine whether an email is spam.
- Controller worker manages statistics and supports a set of commands. For example, it can be used by MDA (e.g. Dovecot) to teach rspamd whether an email is spam when a user moves the email from/to the junk IMAP folder. It communicates with a fuzzy storage worker to save hashes of emails.
- Fuzzy storage worker stores fuzzy hashes of emails in the database. For example, in Redis.
Rspamd has plenty of modules. Some of them are used by default, for example, SPF, DKIM, RBL modules, etc. Some of them should be configured manually. See the rspamd config to find out what modules are used.
apk add rspamd rspamd-clientCreate a directory for rspamd.
mkdir /run/rspamd
chown rspamd:rspamd /run/rspamd/etc/rspamd/local.d/classifier-bayes.conf:
# Store Bayesian statistics in Redis.
backend = "redis";
# Enable autolearning
autolearn = true;/etc/rspamd/local.d/fuzzy_check.conf:
rule "local" {
# Hashing algorithm.
algorithm = "mumhash";
# List of fuzzy hash workers used to check or train.
servers = "0.0.0.0:11335";
# The default symbol applied for a rule.
symbol = "LOCAL_FUZZY_UNKNOWN";
# Set of mime types to check with fuzzy.
mime_types = ["*"];
# Maximum global score for all maps combined.
max_score = 20.0;
# To allow learning for this fuzzy rule, set "no".
read_only = no;
# Ignore flags that are not listed in maps for this rule.
skip_unknown = yes;
# Whether to check the exact hash match for short texts where fuzzy algorithm
# is not applicable.
short_text_direct_hash = true;
# Minimum length of text parts in words to perform fuzzy check.
min_length = 64;
# Symbol -> data for flag configuration.
# `max_score` is a maximum score for this flag. `flag` is an ordinal flag number.
fuzzy_map = {
LOCAL_FUZZY_DENIED {
max_score = 20.0;
flag = 11;
}
LOCAL_FUZZY_PROB {
max_score = 10.0;
flag = 12;
}
LOCAL_FUZZY_WHITE {
max_score = 2.0;
flag = 13;
}
}
}/etc/rspamd/local.d/fuzzy_group.conf:
# Max value for fuzzy hash when weight of symbol is exactly 1.0.
# If value is higher, then the score is still 1.0)
max_score = 12.0;
symbols = {
"LOCAL_FUZZY_UNKNOWN" {
weight = 5.0;
description = "Generic fuzzy hash match";
}
"LOCAL_FUZZY_DENIED" {
weight = 12.0;
description = "Denied fuzzy hash";
}
"LOCAL_FUZZY_PROB" {
weight = 5.0;
description = "Probable fuzzy hash";
}
"LOCAL_FUZZY_WHITE" {
weight = -2.1;
description = "Whitelisted fuzzy hash";
}
}/etc/rspamd/local.d/logging.inc:
filename = "/var/log/rspamd.log";
level = "warning";/etc/rspamd/local.d/redis.conf:
servers = "/var/run/redis/redis.sock";
password = "secret";/etc/rspamd/local.d/replies.conf:
# Apply the given action to emails identified as replies.
action = "no action";
# The records will expire after this time period.
expire = 7d;
# String prefixed to keys in Redis.
key_prefix = "rr";
# The message.
message = "Message is reply to one we originated";
# Symbol yielded on emails identified as replies.
symbol = "REPLY";
# List of Redis servers to use.
backend = "redis";/etc/rspamd/local.d/worker-controller.inc:
# The unix socket used by Dovecot to teach rspamd whether an email is spam.
bind_socket = "/run/rspamd/rspamd-controller.sock mode=0666";/etc/rspamd/local.d/worker-fuzzy.inc:
# The fuzzy worker does not support a unix socket.
bind_socket = "*:11335";
# Enable the fuzzy worker.
# See https://github.com/rspamd/rspamd/issues/4677
count = 1;
# Store fuzzy hashes in Redis.
backend = "redis";/etc/rspamd/local.d/worker-normal.inc:
# Disable the normal worker to free up system resources as it's not
# necessary in the self-scan mode.
enabled = false;/etc/rspamd/local.d/worker-proxy.inc:
# When the milter mode is enabled, the proxy communicates exclusively in
# the milter protocol.
milter = yes;
# The unix socket to communicate with Postfix.
bind_socket = "/run/rspamd/rspamd-proxy.sock mode=0666";
upstream "local" {
default = yes;
# The proxy worker will handle all the spam filtering by itself without
# normal workers.
self_scan = yes;
}/home/virtual/sieve/active.sieve:
require "fileinto";
if header :is "X-Spam" "Yes" {
fileinto "INBOX/Junk";
}/home/virtual/sieve/learn-ham.sieve:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.mailbox" "*" {
set "mailbox" "${1}";
}
if string "${mailbox}" "INBOX/Trash" {
stop;
}
if environment :matches "imap.user" "*" {
set "username" "${1}";
}
pipe :copy "learn-ham.sh" [ "${username}" ];/home/virtual/sieve/learn-ham.sh:
#!/bin/sh
HOST="/run/rspamd/rspamd-controller.sock"
rspamc -h "$HOST" learn_ham/home/virtual/sieve/learn-spam.sieve:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
if environment :matches "imap.user" "*" {
set "username" "${1}";
}
pipe :copy "learn-spam.sh" [ "${username}" ];/home/virtual/sieve/learn-spam.sh:
#!/bin/sh
HOST="/run/rspamd/rspamd-controller.sock"
rspamc -h "$HOST" learn_spamMake bash scripts used by sieve executable.
chown -R virtual:virtual /home/virtual/sieve
find /home/virtual/sieve -type f -iname "*.sh" -exec chmod 710 {} \;Compile all sieve scripts.
sievec /home/virtual/sieveAdding a new domain
For example, you want to add a new domain domain.com.
Add it to the database.
INSERT INTO domains (name) VALUES ('domain.com');Create the DKIM key for domain.com (see the content of the script above).
sh /etc/opendkim/gen-dkim-key.sh domain.com
# mail._domainkey IN TXT ( "v=DKIM1; k=rsa; s=email; "
# "p=<key>" ) ; ----- DKIM key mail for domain.comAdd DNS records. If you are migrating from another mail server, then first create all the mailboxes (see below).
domain.com. IN MX 10 mail.server.com.
domain.com. IN TXT "v=spf1 redirect=_spf.server.com"
mail._domainkey.domain.com. IN TXT "v=DKIM1; h=sha256; k=rsa; s=email; p=<key>"
_dmarc.domain.com IN TXT "v=DMARC1; p=none"Adding a new mailbox
For example, you want to add a new mailbox support@domain.com to an existing domain.
Generate an encrypted password.
doveadm pw -s ARGON2ID
# Enter new password:
# Retype new password:
# {ARGON2ID}$argon2id$<hash>Add a new mailbox support to the database.
INSERT INTO mailboxes (domain_id, name, password_hash)
VALUES (1, 'support', '{ARGON2ID}$argon2id$<hash>');Dovecot reads the hash method placed in front of the password in curly brackets. If all the passwords are stored in the same format, you can set the default password scheme in default_pass_scheme and can not specify it in curly brackets. In this case, Dovecot looks at the password prefix first and, if a scheme is not specified, uses the default scheme.
Migration
You can use imapsync to move all of your emails from previous mail server to a new one using IMAP. So you can move all of your emails, for example, from Gmail to your mail server in one command.
imapsync --host1 mail.source.com --user1 user --password1 "password" \
--host2 mail.target.com --user2 user --password2 "password"You can create a bash script to call this command for all mailboxes.
imapsync can not only just copy all the emails, but also synchronize them between 2 mail servers. For example, you can incrementally migrate a mailbox without finding duplicate emails on the target mail server.
There is also online version of imapsync which allows to copy/synchronize emails between 2 mailboxes (do not forget change your passwords temporarily).
Backups
I don't know best practices to create backups, but I like to think about what could go wrong and then how to avoid it by making backups.
I want to avoid 2 problems:
- The SSD/HDD that holds the mail data has been damaged. To prevent data loss, we can copy all the mail data once a day somewhere else (for example, a to cloud storage), and keep only one copy there.
- Someone (or our own script) with access to the server accidentally ran the wrong command by mistake (or not by mistake), which leads the mail data has been modified incorrectly. If you copy all of the data somewhere once a day, but only notice that the data has been changed incorrectly only after a few days, you could lose the mail data. To avoid it, we can archive all the mail data once a week, send it somewhere else (for example, to a cloud storage), and keep a few backups (for example, 4 backups for the last 4 weeks).
The implementation is very simple. In the first case, you can use rsync or rclone. For example, like this:
rclone sync "/srv/vmail" "google-cloud:my-bucket/mail"Do not forget to create a rclone config.
In the second case, just create an archive and send it somewhere.
tar -czf "/tmp/mail-$(date +%Y%m%d-%H%M).tar.gz" "/srv/vmail"Testing
All the logs are stored in /var/log/postfix.log, /var/log/dovecot.log, and /var/log/rspamd.log. Check the logs and make sure everything works correctly.
Try to send an email locally like this:
telnet localhost 25
# Trying ::1...
# Connected to localhost.
# Escape character is '^]'.
# 220 mail.domain.com ESMTP Postfix
ehlo localhost
# 250-mail.domain.com
# 250-PIPELINING
# 250-SIZE 10240000
# 250-ETRN
# 250-STARTTLS
# 250-ENHANCEDSTATUSCODES
# 250-8BITMIME
# 250-DSN
# 250-SMTPUTF8
# 250 CHUNKING
mail from: <support@domain.com>
# 250 2.1.0 Ok
rcpt to: <support@domain.com>
# 250 2.1.5 Ok
data
# 354 End data with <CR><LF>.<CR><LF>
# Subject: Who is that?
#
# Hey!
# .
# 250 2.0.0 Ok: queued as CDB634C1D91
quit
# 221 2.0.0 Bye
# Connection closed by foreign host.Test different cases: when an incorrect domain is specified, when the domain is correct, but the user does not exist, when everything is correct, when a user exceeded the quota, when a user moves an email from/to the junk IMAP folder, etc.
Send an email to:
- check-auth@verifier.port25.com to make sure SPF and DKIM work correctly and to see how your mail server send emails (you'll receive a reply with a report).
- abuse@mxtoolbox.com to make sure your mail server does not appear to be on any blacklists. The report also contains information about SPF/DKIM/DMARC.
- address shown on the page mail-tester.com to see the recommendations how to increase your chances of getting your emails into the inbox, not spam.
Check here whether the IP address of your server, email or domain is enlisted in one of the blacklist databases. It's the best checker I've ever seen. When mxtoolbox.com and other checkers said that server's IP address has not been enlisted in any of such databases, my outgoing emails sent from this server always ended up in spam in Gmail. Only this tool said that this IP address is listed in one of the blacklist databases. In such a case, changing the IP address solves the problem.
That's all. Hope your mail server will work well.