I this article, I'll describe how I've set up my own mail server using Postfix and Dovecot, added anti-spam protection using Rspamd, and set up SPF/DKIM/DMARC so that outgoing emails get into the inbox, not spam. As a result, you'll have your own mail server where you can add an unlimited number of domains and mailboxes.
In the previous article, we discussed how the process of delivering emails takes place. If you don't know what MTA, MDA, MRA are and what they do, read the article how email works before moving on.
During the set up, I decided to record here for myself why I've done everything this way and not in another way. If you have a good experience in setting up mail servers and you know how to make something much better, you can write me about it by email which you can find at the bottom of this page.
The process of setting up a mail server is strongly depends on what will it be used for. I need a mail server to manage all of the mailboxes in different domains of my own projects (support@domain.com, etc).
Let's get started. First, we should choose a Mail Transfer Agent (MTA) and Mail Delivery Agent (MDA).
Choosing agents
Mail Transfer Agent
According to a Security Space study conducted on November 1, 2023 (MTA was detected on 432467 servers), Exim is used on 58.73% of mail servers (254000), Postfix – on 34.86% (150774), and a few dozen other MTA are used by the remaining percentages of servers. You may notice that in 2023, the number of Exim users was reduced (60.90% in 2022, 58.73% in 2023), but the number of Postfix users was increased (32.49% in 2022, 34.86% in 2023). Maybe they moved from Exim to Postfix :) Anyway, Exim is used 2 times more often than Postfix, and the rest of the MTA are used extremely rarely, so we will consider only them.
Here are the main points what I could find about Exim and Postfix:
- Postfix has a high queue speed compared to Exim, while less CPU usage.
- Exim configuration is more flexible than Postfix have. Exim has its own language that allows you to write scripts for processing emails.
Due to the work speed and safety, I decided to use Postfix.
Mail Delivery Agent
According to the Open Email Survey conducted in 2020 (IMAP server was detected on 3674586 servers), the most popular IMAP server is Dovecot (76.93%), followed by Courier (7.95%), and a few dozen other IMAPs are used by the remaining percentages of servers. Over the past 8 years, Dovecot (48.37% in 2012, 76.93% in 2020) has been used much more often than Courier (30.70% in 2012, 9.06% in 2020).
Here are the main points what I could find about Dovecot and Courier:
- Many people note that Dovecot uses much less disk I/O operations compared to Courier due to the availability of high-performance indexing and caching. They says that the number of I/O operations may differ by 10 times.
- Dovecot uses less RAM and CPU compared to Courier.
- Dovecot is praised for its fast work on mailboxes with about 30 folders with ~8000 emails per folder and ~200 new messages each daily. In comparison, Courier was very slow.
- Dovecot has the Full Text Search to search for emails, which is not available in Courier.
There is also Cyrus, but it's used much less often (only 0.58% of servers according to the survey). People says it has almost the same performance (I/O operations, both have FTS to search for emails, etc). If it's important for you, you can make a benchmark of Cyrus-IMAP and Dovecot. More about it here. Cyrus supports for calendar and contact synchronization, but the same is achievable using the separate package with Dovecot, like Horde Groupware. It allows users manage and share their calendars, contacts, tasks, notes, and files.
Timo Sirainen, as he said, instead of evaluating features of IMAP servers, started to evaluate their source code and he didn't like any of them. As a result, he developed Dovecot, which nowadays is the most popular IMAP server all around the world. Peer Heinlein, the author of this book with the foreword from Timo Sirainen, said: "Since version 2.x, Dovecot has been far superior to all the other candidates. Courier has no right to exist any more, and Cyrus is not much better. Dovecot versions 2.2.x provide options that Cyrus admins would not dare to dream of." He mentioned the following advantages of Dovecot:
- The speed is faster.
- Scope of functions is far wider.
- The config clearer and more flexible.
- The log file is easy to read.
- RFC-conform (Cyrus is not).
- Properly programmed. In 2017, Mozilla conducted a security audit of the Dovecot code and said that they were extremely impressed with the quality of the dovecot code, writing that "despite much effort and thoroughly all-encompassing approach, the Cure53 testers only managed to assert the excellent security-standing of Dovecot".
- Basic config allows to make plenty of performance tuning and improvements.
- The mdbox storage format provides a storage system that scales well even for millions of accounts, but also the use of slower hard drives or even object storage.
- Allows to implement active/active clusters without great effort.
Because of the many reasons described above, I decided to choose Dovecot.
I think in addition to the official documentations I had read almost all the articles I could find about setting up a mail server using Postfix and Dovecot, yet I still didn't understand how everything works, or why it didn't work sometimes. It wasn't until I read two great books about Postfix and Dovecot. It turns out that it's pretty straightforward once you know the basics. So there is a theory at first to help you understand everything, and then there are the configs. Hope this article will help you to set up your own mail server. Hope it will also help me remember everything quickly in the future (actually, that's why I'm writing it).
Postfix
Postfix is a Mail Transfer Agent. Broadly speaking, it receives inbound and delivers outbound emails via the SMTP protocol.
When an MTA receives an inbound email, it either hands these emails off to a Mail Delivery Agent for the final delivery to a user, or forward them to one or several other emails, or relay them to another MTA. Postfix can also save an email to a mailbox by itself, but it doesn't handle IMAP/POP3 communications with MUA (MDA [Dovecot] does it).
If an MTA cannot deliver an email or pass it along, MTA:
- Rejects the email if an error happens during the SMTP communication. For example, if the recipient does not exist.
- Bounces the email to the sender if it has been already accepted. MTA creates a new email with an error, for example, when a user's mailbox is full and sends it to the sender.
- Sends an email to an administrator.
When an MTA delivers an outbound email, it reads the hostname of the recipient, determines the IP address of the next mail server using DNS, and sends that email via SMTP.
Different parameters in the Postfix config are setting up different programs (parts of Postfix), so first we should figure out which programs exist and how they work.
How Postfix works
Postfix performs many tasks. Each task is performed by a separate program. When we run Postfix using postfix start, the master daemon is started and keeps running until we stop it by postfix stop. This daemon invokes other programs which process their tasks and terminate.
Each program operates with the minimum privilege necessary to accomplish their tasks. If you want Postfix will be just a relay, you can turn off, for example, a program which delivers messages to a user's mailbox for safety. Thus, even if an attacker will gain access to one of the programs, he cannot do more than this program can.
There are 3 main types of programs in Postfix:
- Programs that receive emails and hand them off to the queue manager.
- The queue manager which calls another programs to deliver emails.
- Programs that deliver emails in different ways.
We can send an email to Postfix either locally, or via a network.
If we send an email locally (for example, using sendmail), first, the postdrop command will deposit it in the maildrop queue. It works even when Postfix is not started. All postfix queues are usually located in the /var/spool/postfix directory, so we can see maildrop queue in this directory. Let's see how it works.
ls /var/spool/postfix/maildrop/
sendmail root@localhost
# Subject: Test
# Test message
ls /var/spool/postfix/maildrop/
# 0C2594C048BThen the pickup daemon picks up that email, makes sanity checks and hands it off to the cleanup daemon.
The cleanup daemon in conjunction with the trivial-rewrite daemon inserts missing message headers, converts addresses to the correct format, and translates addresses based on different lookup tables (canonical, etc). Read more here. Finally, the cleanup daemon deposits the cleaned-up email in the incoming queue and notifies the queue manager qmgr.
The qmgr uses trivial-rewrite to determine the transport method to use, the host for delivery, and the recipient's address. If system resources are available, qmgr moves the email from the incoming queue to the active queue and invokes the appropriate delivery agent (smtp, lmtp, local, virtual, pipe) to send the email to the final destination or relay it to the next destination.
If we send an email via network using the SMTP protocol, the smtpd daemon will accept it, make sanity checks and hand it off to the cleanup process. The rest process is the same.
If a delivery agent temporarily can't receive the email (returns 4xx, or there is a temporary DNS problem), qmgr deposits the email to the deferred queue and marks it with a timestamp when the next delivery attempt should occur. Periodically (every 300 seconds by default), the queue manager scans the deferred queue and checks that timestamp. When that time has already come, qmgr moves that email from the deferred queue to the active queue for the next delivery attempt. After configured period of time (5 days by default), Postfix gives up trying to deliver the email and bounces it back to the sender.
If a delivery agent returns 5xx (permanent problem), the bounce daemon creates a new email with an error, and hands it off to the cleanup daemon which send it to the incoming queue for a delivery.
The queue manager also works with defer and bounce daemons to save the reason why an email has been deferred or should be bounced. Such information is stored in the /var/spool/postfix/defer and /var/spool/postfix/bounce directories, respectively. defer and bounce daemons use the information stored in these directories to generate emails with errors.
Postfix can send emails with errors to an administrator. By default, Postfix reports only when an email could not be delivered because of system resource problems or software problems, but it's possible to report another types of errors. Read more here.
You can dive deeper into Postfix architecture in the official documentation.
Postfix address classes
Using different address classes, Postfix determines which email to accept, and how to deliver it. Depending on the address class, the queue manager invokes the appropriate delivery agent to handle the email.
Each address class consists of 3 parts:
- The list of domains (only emails sent to these domains will be accepted).
- The list of recipient addresses (only emails sent to these addresses will be accepted). If not specified, Postfix accepts any address.
- The delivery transport (how to deliver emails).
There are 4 main address classes: local, virtual alias, virtual mailbox, and relay.
Local address class
When the local address class is used, Postfix deposits emails into the local mailboxes for users, who have a UNIX system account. Emails for different domains, but the same name, go to the same mailbox. The default transport is the local delivery agent.
/etc/postfix/main.cf:
mydestination = $myhostname, $mydomain, /etc/postfix/mydestinations
local_recipient_maps = lmdb:/etc/postfix/local_recipients
local_transport = local/etc/postfix/mydestinations:
domain1.com
domain2.com/etc/postfix/local_recipients:
kate -
john -postmap /etc/postfix/local_recipientsVirtual alias address class
When the virtual alias address class is used, Postfix forwards emails to other addresses in other address classes.
/etc/postfix/main.cf:
virtual_alias_domains = /etc/postfix/virtual_alias_domains
virtual_alias_maps = lmdb:/etc/postfix/virtual_aliases
canonical_maps = lmdb:/etc/postfix/canonical/etc/postfix/virtual_alias_domains:
domain1.com
domain2.com/etc/postfix/virtual_aliases:
kate@domain1.com kate.brown
kate@domain2.com kate.smith
john@domain2.com john/etc/postfix/canonical:
kate.brown kate@domain1.com
kate.smith kate@domain2.com
john john@domain2.compostmap /etc/postfix/virtual_aliases
postmap /etc/postfix/canonicalVirtual mailbox address class
When the virtual mailbox address class is used, Postfix sends emails to mailboxes specified in the virtual_mailbox_maps config. Users do not need to have a UNIX system account. The default transport is the virtual delivery agent.
/etc/postfix/main.cf:
virtual_mailbox_domains = /etc/postfix/virtual_mailbox_domains
virtual_mailbox_maps = lmdb:/etc/postfix/virtual_mailboxes
virtual_transport = virtual
virtual_mailbox_base = /usr/local/vmail
virtual_uid_maps = static:1000
virtual_gid_maps = static:1000/etc/postfix/virtual_mailbox_domains:
domain1.com
domain2.com/etc/postfix/virtual_mailboxes:
kate@domain1.com domain1.com/kate.brown/
kate@domain2.com domain2.com/kate.smith/
john@domain2.com domain2.com/john/The key is an email address, and the value is a relative path to the mailbox. Slash at the end of the path means that the maildir mailbox format will be used. Remove slash at the end to use the mbox format (not recommended).
postmap /etc/postfix/virtual_mailboxesRelay address class
When the relay address class is used, Postfix relays emails to a different system. It is used when users' mailboxes are managed by the different system. The default transport is relay (clone of the smtp delivery agent).
Examples:
- MDA manages the mailboxes, not Postfix, even if it's located on the same machine. It's better than if Postfix saves the emails by itself because MDA usually supports quota rules, sieve filters, etc. out of the box. Also you might want to use a proprietary mailbox storage supported only by MDA, for example, mdbox.
- A backup MX server. If the primary mail server is not available, the backup mail server receives emails and queued them until the primary mail server will be ready to accept them. See implementation here.
- There are multiple departments. Each one has its own Postfix on its own machine. They shouldn't be accessible from the outside, but they manage users' mailboxes (for example, they can use virtual mailbox address class). In this case, the main Postfix should be configured as a relay and forward emails to the departments.
/etc/postfix/main.cf:
relay_domains = /etc/postfix/relay_domains
relay_recipient_maps = lmdb:/etc/postfix/relay_recipients
relay_transport = relay
transport_maps = lmdb:/etc/postfix/transport/etc/postfix/relay_domains:
domain1.com
domain2.com/etc/postfix/relay_recipients:
kate@domain1.com -
kate@domain2.com -
john@domain2.com -/etc/postfix/transport:
domain1.com lmtp:[127.0.0.1]
domain2.com relay:[mail2.server.com]postmap /etc/postfix/relay_recipients
postmap /etc/postfix/transportDefault address class
If the domain is not specified in one of the address classes, the email will be sent by default transport to the next destination (Postfix determines where to send the email though DNS lookups). The default transport is the smtp delivery agent.
To prevent Postfix from being an open relay, make sure you have reject_unauth_destination in smtpd_recipient_restrictions.
/etc/postfix/main.cf:
default_transport = smtp
smtpd_recipient_restrictions = reject_unauth_destinationCustom address class
There are other delivery agents which you can use (must be configured in the master.cf file). For example, using the pipe daemon, Postfix can deliver emails to an external command.
Let's imagine we have the email address support@domain.com where our clients send emails to. We also have an email address for each of our manager who answers to these emails: manager1@domain.com and manager2@domain.com. We can create a script which send all emails to one of the managers using the round-robin algorithm. Thus 50% emails sent to support@domain.com will be sent to the first manager, and the rest 50% – to the second manager.
Different address classes can be mixed if you want to handle different domains in different ways.
How to configure Postfix
The /etc/postfix directory has 2 configuration files: master.cf and main.cf. These files should have permission 644 (anyone can read, owner can write) and be owned by root. It's by default, so do not change the permission and the owner of these files.
The master.cf file contains a list of daemons launched by the master daemon. Each line represents a daemon and how it should be run. You can find what does each column mean at the beginning of the file. Read more here. The main.cf file contains default configurations of these daemons, but you can override them directly in the master.cf file using an -o option like this -o smtpd_tls_auth_only=yes.
The main.cf file contains all the configuration parameters (parameter = value). The value can be one of:
- String: mydestination = domain.com. Quotation marks considered part of the value, so do not use them.
- Reference: mydestination = $mydomain.
- Pointer to a file: mydestination = /etc/postfix/mydestinations.
- Pointer to a lookup table: mydestination = lmdb:/etc/postfix/mydestinations.
- The list of values, even the list of lookup tables.
See here what parameters support what types of values.
The format of master.cf, main.cf, lookup tables, and alias files is as follows:
- Empty lines and lines that starts with # are ignored.
- A line that starts with whitespace (tabs or spaces) continues the previous line.
Lookup tables are indexed files that provide faster access to the stored items. A lookup table stores key/value pairs. Keys are case-insensitive and must be unique. After any changes it's necessary to create an indexed version of the file using postmap <path>. The command postmap -q <key> <path> allows to query a lookup table. The reference to a lookup table should have the format parameter = type:name, where type is a lookup table type, and name is the resource containing keys and values (for most types name is the file path). The file path shouldn't contain file extension. In cases when you need to store only keys in a lookup table, use any text as a value, for example, a comment. A lookup table should be used when you want to store thousands of items, otherwise use a simple text file.
The example of the lookup table.
/etc/postfix/canonical:
name@domain.com fullname@domain.com/etc/postfix/main.cf:
canonical_maps = lmdb:/etc/postfix/canonicalAlias files use a Sendmail-compatible format. To build the alias database from the text file use postalias <path>. newaliases rebuilds alias files specified in the alias_database parameter.
The example of the alias file.
/etc/postfix/aliases:
# Email address (one or many)
alias1: manager1@domain.com, manager2@domain.com
# The path to a file where new emails are appended
alias2: /usr/local/mail/alias2_mailbox
# Command
alias3: "|/usr/local/bin/autoreply"
# Include a list of alias targets
alias4: :include:/usr/local/mail/alias4_list/etc/postfix/main.cf:
# Point to the file (one or many)
alias_maps = lmdb:/etc/postfix/aliases
# Should be also set because it's used by `newaliases`
alias_database = lmdb:/etc/postfix/aliasesTo restrict mail delivery to external commands and files use allow_mail_to_commands and allow_mail_to_files.
You can manage the main.cf file using the postconf command.
postconf myhostname # Get
postconf myhostname=mail.domain.com # Set
postconf -m # List the names of all supported lookup table typesPostfix selects logging to syslogd by default. You can set a file where logs should be written.
maillog_file = /var/log/postfix.log
# Select logging to stdout. Postfix must be started with `postfix start-fg`.
# maillog_file = /dev/stdoutTo apply new changes, Postfix must be reloaded. Postfix gracefully terminates running processes after they have finished any tasks they are working on, rereads its configuration files, and continues to receive mail without interruption.
postfix reloadTo check configurations, run the following command. It checks the configuration problems, looks at directory and file ownership, and creates any missing directories. If everything is correct, there will be no messages.
postfix checkDNS
Let's say the IP address of your server is 12.34.56.78 and domain is mail.server.com. You want to receive emails sent to the domain domain.com (for example, to support@domain.com).
To send an email, MTA (the smtp delivery agent in Postfix) makes at least 2 requests for DNS:
- To get the hostname of recipient's MTA using the MX record.
- To get the IP address of that hostname using the A record.
So the domain of the mail server must have at least the A record mail.server.com. IN A 12.34.56.78. It's also highly recommended to set the PTR record 78.56.34.12.in-addr.arpa. IN PTR mail.server.com. so another mail servers can check the hostname of your server before receiving emails from you to prevent spam. Otherwise, your mail server cannot send emails to any mail servers or your emails might be marked as spam. The PTR record is set on the server provider's side. For example, in Hetzner, open your server, go to Networking, and set the reverse DNS to mail.server.com.
The domain domain.com with mailboxes must have at least one MX record domain.com. IN MX 10 mail.server.com.. According to RFC 974, the MX record should be a hostname, not an IP address. It should also not be an alias (CNAME record) because it might cause a mail loop.
You should also set SPF and/or DKIM records so other mail servers can receive your emails. For example, gmail requires all senders to authenticate with either SPF or DKIM (both are better), otherwise you can't send emails to gmail.
An SPF record provides a list of legitimated IP addresses as being permitted or not permitted to use the domain for sending emails. If someone sends an email from an invalid IP address, it will be considered as spam and can be rejected. To avoid changing valid IP addresses in each domain all the time the list of IP changes, first create the SPF record (TXT) v=spf1 a:mail.server.com ~all for the domain _spf.server.com (used to check MAIL FROM), and then create the SPF record (TXT) v=spf1 redirect=_spf.server.com for the domain domain.com. If the list of IP addresses will be changed, just update the SPF record for _spf.server.com. Also set v=spf1 a ~all for the domain mail.server.com (used to check HELO/EHLO). Read more in RFC 7208.
A DKIM record stores the public key used to verify that an email has been sent from the domain and has not been modified. The mail server before sending an outgoing email, creates the DKIM signature using the private key, adds it to the email, and send this email to another mail server which check this signature using the public key specified in the domain's DKIM record (TXT). Read more in RFC 6376. The implementation see here.
A DMARC record specifies the actions to be taken when an email fails authentication checks, such as SPF and DKIM. The DMARC record should be set for _dmarc.domain.com as a TXT record. Examples: v=DMARC1; p=none; fo=1; rua=mailto:dmarc-feedback@domain.com; ruf=mailto:auth-reports@domain.com (if you want to get reports), v=DMARC1; p=none (if not). Read more in RFC 7489.
As a result, you need to set the following DNS records:
mail.server.com. IN A 12.34.56.78
78.56.34.12.in-addr.arpa. IN PTR mail.server.com
_spf.server.com. IN TXT "v=spf1 a:mail.server.com ~all"
mail.server.com. IN TXT "v=spf1 a ~all"
domain.com. IN MX 10 mail.server.com.
domain.com. IN TXT "v=spf1 redirect=_spf.server.com"
mail._domainkey.domain.com. IN TXT "v=DKIM1; k=rsa; p=<key>"
_dmarc.domain.com IN TXT "v=DMARC1; p=none; fo=1; rua=mailto:dmarc-feedback@domain.com; ruf=mailto:auth-reports@domain.com"It's a good idea to have multiple MX records. The first one should point to the primary server, and the rest to secondary servers which receive inbound emails and hold them until the primary server will be ready to accept them.
Postfix retrieves all of the MX records, sorts them by priority in ascending order, and at the beginning tries to send an outbound email to the first one. If the first MTA cannot receive the email, Postfix tries to send it to the next one (see smtp_skip_4xx_greeting and smtp_skip_5xx_greeting to change this behavior). If there are no MX records for a domain, Postfix checks the A record and tries to deliver the email by IP address.
To receive emails for a domain, you should not only set the MX record, but also add this domain to Postfix in one of the address classes (see above). Do not specify a domain in more than one of address classes.
Dovecot
If an MTA like Postfix is responsible for receiving and sending emails via SMTP (sometimes for depositing them in mailboxes as well), an MDA/MRA like Dovecot is used to save emails in mailboxes, manage them (folders, quotas, etc.), and retrieve emails from them via POP3/IMAP. Postfix can receive emails and hand them off to Dovecot using LMTP, thus Postfix will be just a front relay.
The LMTP protocol is almost the same as SMTP, but it's designed only for local use. After the DATA command, SMTP returns a single status message indicating whether the email has been delivered. If an email cannot be saved successfully in one of mailboxes, the email will be rejected for all recipients. In contrast, the LMTP protocol returns individual status messages for each recipient after the DATA command. If an email cannot be saved successfully in a mailbox for a specific recipient, the email will be rejected for that recipient alone, without affecting the delivery to other recipients.
There is also LDA (Local Delivery Agent), but it's not recommended to use it because:
- LDA receives emails via pipe, so it can't return errors back to Postfix.
- LDA is started every time an email is received. So it will work slower than if you use a daemon, like LMTP.
- An email with multiple recipients will be handed off to LDA multiple times, for each recipient individually.
- LDA does not support for dynamic recipient validation (for example, if you are storing users in /etc/dovecot/users).
- If you create UID/GID for each user, there might be a problem with permissions, because Postfix runs processes with the minimum privileges, not with root.
Authentication
Dovecot makes authentication requests when a user logs in or an email is received.
When a user logs in, Dovecot uses passdb to verify his password and decides whether to grant that user access. After that, Dovecot uses userdb to determine UID/GID under which the email files are saved, and the path to his home directory where email files, sieve scripts, and other individual items are stored.
When an email is received, Dovecot uses only userdb to determine where the email should be saved (home directory), and which file permissions should be used (UID/GID).
You can make userdb and passdb lookups using doveadm CLI tool. It might be useful during testing, especially when PostgreSQL/MySQL is used to store these data.
doveadm user my_user # userdb lookup
doveadm auth test my_user # passdb lookupThere are many ways how you can store userdb and passdb. The most popular are SQL, LDAP, and passwd-file. SQL allows to store the data in PostgreSQL, MySQL, or SQLite. passwd-file allows to store the data in one plain text file (for example, in /etc/dovecot/passwd-file). As Peer Heinlein said, once he used passwd-file even for storing more than 100000 users, and it took just a few milliseconds to authenticate a user because of caching in Dovecot.
Dovecot uses UID (User ID) and GID (Group ID) to manage user's access to mailboxes. Usually, it's better to use the same UID for all users because:
- Easier to write scripts that should have an access to mailboxes. For example, to restore a backup.
- Easier to implement shared folders.
- You can have less than 2^16 UIDs for Linux kernel version <=2.4, and 2^32 for version >2.4.
- Different UIDs gives only a little security benefits because Dovecot has access to all user's email data anyway.
There are 2 authentication mechanisms: plaintext (PLAIN, LOGIN) and non-plaintext (SCRAM-SHA-256, etc.).
PLAIN and LOGIN are both transmit a password in plain text (should be used only with SSL/TLS), but there is a slight difference. PLAIN (RFC 4616) transmits the user name and password in a single line, LOGIN (Microsoft) - in two base64-encoded lines. In such a case, you can store all the passwords encrypted in a database (for example, using ARGON2ID) because even if an attacker gains access to that database, he won't be able to read passwords. It makes sense because a lot of users use the same password everywhere and there is nothing you can do about it, so it will be not complicated for an attacker to write a script to try the same passwords in other mail servers (gmail, yahoo, etc.). Thus, it's better to store all the passwords encrypted in a database.
Non-plaintext mechanisms are usually used when users can log in without an SSL/TLS-secure connection. This approach works as follows. The server generates a unique session key for each login. The client and server then use this key to generate a hash value from the password. This key is valid only for that particular session. In such a case, you should store all the passwords in plain text in a database.
By default Dovecot makes 2 requests during user authentication: passdb and userdb. To avoid making the second request, the passdb request should return not only user (or username, domain) and password, but also userdb_home, userdb_uid, and userdb_gid. This is called prefetching. The userdb request will still be used when an email is received or the shared folder is being accessed. See the implementation in the second part of this article.
Mailbox storage formats
There are 3 storage formats in Dovecot: mbox, Maildir and mdbox.
The mbox format stores all emails for one IMAP folder in one file. This format is designed for use with POP3, where emails are deleted regularly on the mail server after fetching them to avoid increasing a file size constantly. But when IMAP is used, emails usually remain on the mail server and are rarely deleted. It leads to problems, for example, when you delete an email from the middle of 1GB file, the entire file has to be copied. Also only one process can manage that file at a time (add a new email, delete an email, etc.). As Peer Heinlein said: "In some cases, Postfix was unable to write a new message to the mbox file for 3-4 hours, because the file was constantly occupied and locked by other processes.". Don't use mbox.
The Maildir format stores each email in one file. An IMAP folder is a directory with the email files in it. Adding a new email it's just creating a new file, deleting - the same. So different processes can work with the same mailbox simultaneously. But a huge number of small email files on a disk (hundreds of thousands) can degrade the performance of making backups.
The mdbox format stores multiple emails in one file of several MB in size (~ 100 emails per file). So creating backups is much faster, delete operation is not a problem because the file is only a few MB in size (see mdbox_rotate_size, it's recommended to set 5-10 MB). Actually, when a user delete an email, it is not deleted from a file, it's just marked as deleted in the Dovecot index, and your cron script should run doveadm purge -A, for example, every weekend at night to physically delete all emails marked as deleted. Different process can also manage different files in parallel. The disadvantage is that you cannot restore/delete emails and create/delete IMAP folders manually, you should use the doveadm command for that. Deleting one of dovecot.index.* files (stores keywords, IMAP flags, etc) would damage the mdbox storage.
Peer Heinlein gave a good recommendation which storage format to choose. Use Maildir until you have performance issues (especially with backup process) and if your Maildir volume is less than 3-4 TB in size. It's not a big problem to migrate from Maildir to mdbox when that time come. So if you are starting from scratch and don't have TBs of email data, start from Maildir and use doveadm command in your scripts so you don't have to change them. The example of the migration script you can find in his book about Dovecot.
It's recommended to store all email data for Dovecot using ext4 or xfs filesystem with noatime. It will be several times faster compared to other filesystems.
Now that we know the basics, it's time to put everything into practice. In the second part of this article I'll show you all the configurations of Postfix, Dovecot and Rspamd.