SMTP: MTA and Email Delivery

Why SMTP matters

The protocol itself is simple text, but the surrounding ecosystem is dense: multiple versions, extensions, three ports, anti-spam standards (SPF/DKIM/DMARC), and a three-layer hierarchy (MUA -> MSA -> MTA -> MTA -> MDA -> MUA). You need this when you:

• Set up a corporate mail server (Postfix/Exim for outbound, Dovecot for IMAP)
• Configure a relay host for applications ("send notifications through X")
• Debug why "messages are not arriving" (always DNS + SPF + DMARC)
• Read journalctl -u postfix and understand what the error means

The current trend: internal mail servers are moving to SaaS (Google Workspace, Microsoft 365). Self-hosted SMTP survives for outbound (notifications, alerts) and in environments where cloud is not an option (compliance, government).

Three ports

• Port 25 for outbound from home/ISP networks is often blocked by the carrier (anti-spam). Send through port 587 to your own MTA instead.
• Port 587 is the modern choice for clients. Authentication is required.
• Port 465 was deprecated in 1998 and revived in 2018 (RFC 8314) for clients where STARTTLS starts in plaintext, which makes it vulnerable to a downgrade attack that strips the STARTTLS extension.

An SMTP session (live)

$ telnet smtp.example.com 25

Trying 198.51.100.10...

Connected.

220 mail.example.com ESMTP Postfix

> EHLO client.example.org

250-mail.example.com Hello client

250-PIPELINING

250-SIZE 52428800

250-STARTTLS

250-AUTH PLAIN LOGIN

250 ENHANCEDSTATUSCODES

> MAIL FROM:<sender@example.org>

250 2.1.0 Ok

> RCPT TO:<rcpt@example.com>

250 2.1.5 Ok

> DATA

354 End data with <CR><LF>.<CR><LF>

> Subject: hello

>

> body

> .

250 2.0.0 Ok: queued as ABC123

> QUIT

221 2.0.0 Bye

The protocol is text-based, so you can debug it by hand. EHLO (replacing HELO) requests extensions (ESMTP): STARTTLS, AUTH, SIZE, and others.

DNS: the MX record

For anyone sending mail to example.com, DNS must say which server to use:

example.com.   IN  MX  10  mail.example.com.

example.com.   IN  MX  20  backup-mail.example.com.

• 10, 20 are priorities. Lower value means higher priority. An MTA tries records in ascending priority order.
• Multiple MX records at the same priority get round-robin treatment.

Check with cmd-dig:

dig example.com MX +short

If no MX record exists, mail falls back to the domain's A/AAAA record (RFC 5321 §5.1).

SPF: who may send from your domain

A TXT record in DNS:

example.com.  IN  TXT  "v=spf1 ip4:198.51.100.10 include:_spf.google.com -all"

The receiving MTA looks up the SPF record of the sender's domain and checks whether the connecting IP is on the allowed list. If not, -all (hard fail) rejects the message.

Mechanisms:

• ip4:/ip6:, specific IPs or subnets
• a, any A record of the domain
• mx, any MX target
• include:other.com, inherit SPF from another domain
• ~all (soft fail), mark as suspicious
• -all (hard fail), reject

SPF breaks on forwarding (the forwarder substitutes its own source address, not the original). The fix is DKIM.

DKIM: signing messages

The sending MTA signs outgoing messages with a private key. The public key lives in DNS:

default._domainkey.example.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIGfMA0..."

The receiver reads the DKIM-Signature header, fetches the public key from DNS, and verifies the signature over the headers and body. A valid signature means the message came from that domain and was not modified in transit.

Unlike SPF, a DKIM signature survives forwarding as long as the forwarder does not alter the body.

DMARC: policy for the receiver

DMARC orchestrates SPF and DKIM:

_dmarc.example.com.  IN  TXT  "v=DMARC1; p=quarantine; rua=mailto:dmarc@example.com; pct=100"

• p=none, report only, take no action
• p=quarantine, deliver to spam
• p=reject, discard
• rua=, where to send aggregate reports
• pct=, what percentage of messages the policy applies to (for gradual rollout)

The minimum required setup for outbound mail: SPF + DKIM + DMARC. Without these, Gmail, Outlook, and others will almost certainly classify your messages as spam.

Postfix vs Exim

For a new setup, choose Postfix. Use Exim if you have inherited an existing installation.

Minimal outbound with Postfix

/etc/postfix/main.cf:

myhostname = mail.example.com

mydomain = example.com

myorigin = $mydomain

inet_interfaces = all

inet_protocols = ipv4

mydestination = $myhostname, localhost.$mydomain, localhost

# Relay only from the local network (prevents open relay)

mynetworks = 127.0.0.0/8, 10.0.0.0/24

# TLS server

smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem

smtpd_tls_key_file  = /etc/letsencrypt/live/mail.example.com/privkey.pem

smtpd_tls_security_level = may          # opportunistic

smtpd_tls_loglevel = 1

# TLS client (for outbound)

smtp_tls_security_level = may

smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

postfix check                # check syntax

systemctl reload postfix

postqueue -p                 # view queue

postsuper -d ALL deferred    # clear deferred queue (use with care)

When something goes wrong

• relay access denied: you are sending from an IP not in mynetworks without auth. Either add the IP to mynetworks or require SMTP AUTH on port 587.
• Messages arrive but go to spam: SPF/DKIM/DMARC is missing or misconfigured. Check with https://mail-tester.com or Gmail "Show original".
• Connection timed out on port 25: the carrier blocks outbound port 25. Use port 587 to a relayhost.
• Deferred queue grows: the receiver is greylisting (returns 451 on the first attempt, expects a retry). This is normal for Gmail. Run mailq to see what is stuck.
• Helo command rejected: need fully-qualified hostname: myhostname is not an FQDN, or the PTR record for the IP does not match the EHLO name.
• SSL_connect:error on STARTTLS: one side has an outdated cipher. Set smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1.
• Messages to self never arrive: the local domain is not in mydestination, so the MTA tries to resolve it via DNS, the MX resolves back to itself, and a routing loop forms.