OpenVPN: TLS-based VPN

Why OpenVPN

In 2002, OpenVPN delivered what did not exist before: a VPN over TLS on top of UDP/TCP without IPsec complexity. It remains the standard for:

• Corp VPN with per-user certificates and LDAP/RADIUS authentication
• TCP-443 camouflage (looks like HTTPS, passes through DPI firewalls where WireGuard and IPsec are blocked)
• Site-to-site with push routes and dynamic IP assignment to clients
• L2 bridging via tap mode (when you need a single broadcast domain across the tunnel)

Drawbacks compared to [[wireguard|WireGuard]]:

• Userspace, slower, more CPU usage
• Configuration is more complex, you must generate certificates
• One process per server (not one per peer as in WG)

tun vs tap

tun (L3): VPN sees IP packets        -- default, ~99% of cases

tap (L2): VPN sees Ethernet frames   -- when you need a single broadcast domain

• tun does IP routing between subnets. More efficient (~1500 bytes/frame).
• tap is bridge-in-bridge. Required for DHCP/Wake-on-LAN/non-IP protocols. Client support is weak on mobile (Android/iOS do not support tap).

If you are unsure which to pick, use tun.

Certificates with easy-rsa

apt install easy-rsa openvpn

cd /usr/share/easy-rsa

./easyrsa init-pki

./easyrsa build-ca                              # password for the CA key

./easyrsa gen-dh                                # Diffie-Hellman params

./easyrsa build-server-full vpn.example.com nopass

./easyrsa build-client-full client1 nopass

# tls-crypt key to protect the handshake (not part of easy-rsa, generated by openvpn)

openvpn --genkey secret /etc/openvpn/server/tls-crypt.key

Files:

• pki/ca.crt, the public CA certificate
• pki/issued/<name>.crt, issued certificate
• pki/private/<name>.key, private key
• pki/dh.pem, DH params
• tls-crypt.key, symmetric key for encrypting the handshake

Server config

/etc/openvpn/server/server.conf:

port 1194

proto udp

dev tun

topology subnet

ca       ca.crt

cert     server.crt

key      server.key

dh       dh.pem

tls-crypt tls-crypt.key

server 10.8.0.0 255.255.255.0          # address pool for clients

ifconfig-pool-persist ipp.txt           # remember which IP belongs to which client

push "redirect-gateway def1 bypass-dhcp"   # full-tunnel

push "dhcp-option DNS 1.1.1.1"

push "dhcp-option DNS 8.8.8.8"

keepalive 10 120                        # ping every 10s, timeout 120s

cipher AES-256-GCM

auth SHA256

user  nobody

group nogroup

persist-key

persist-tun

status      /var/log/openvpn-status.log

log-append  /var/log/openvpn.log

verb 3

explicit-exit-notify 1

Start the service:

systemctl enable --now openvpn-server@server

Remember to enable ip-forwarding and [[cmd-iptables|MASQUERADE]] for the pushed full-tunnel to work.

Client config

client.ovpn is a single file with certificates embedded inline:

client

dev tun

proto udp

remote vpn.example.com 1194

resolv-retry infinite

nobind

persist-key

persist-tun

remote-cert-tls server

cipher AES-256-GCM

auth SHA256

verb 3

<ca>

-----BEGIN CERTIFICATE-----

... ca.crt ...

-----END CERTIFICATE-----

</ca>

<cert>

... client.crt ...

</cert>

<key>

... client.key ...

</key>

<tls-crypt>

... tls-crypt.key ...

</tls-crypt>

One file is all you hand to the user. It imports into every client app (Tunnelblick, OpenVPN Connect, NetworkManager).

tls-auth vs tls-crypt

Both protect the handshake from DDoS and passive analysis:

• tls-auth (HMAC): a packet is dropped if it lacks an HMAC signature. The older option.
• tls-crypt (encryption): the handshake is encrypted with this key. From the outside, you cannot even tell it is OpenVPN traffic. Prefer this.

Since 2.5, tls-crypt-v2 supports per-client keys for even stronger protection.

TCP fallback

proto tcp

port 443

This disguises traffic as HTTPS. Drawbacks:

• TCP-over-TCP degrades badly under packet loss (double retransmit)
• Slower than UDP

Use this only when UDP is blocked. You can run both servers on separate ports and have the client try UDP first.

Authentication

• PKI cert-only (default): a client with a private key from a trusted CA is accepted
• PKI + username/password: auth-user-pass-verify backed by PAM/RADIUS/script
• 2FA via TOTP: through the PAM stack (pam-google-authenticator)
• client-cert-not-required: password only (avoid this, it weakens security)

Per-user revocation:

./easyrsa revoke client1

./easyrsa gen-crl                       # generate the CRL

cp pki/crl.pem /etc/openvpn/server/

# in server.conf: crl-verify crl.pem

systemctl reload openvpn-server@server

OpenVPN vs WireGuard vs IPsec

Troubleshooting

• TLS Error: TLS key negotiation failed to occur within 60 seconds: UDP/TCP is not reaching the server. Test with nc -u 1194 or nc -z 443.
• Client connects but no internet: you forgot the redirect-gateway push or MASQUERADE on the server. Check DNS push as well.
• Handshake fails with cipher mismatch: server and client disagree on the cipher. The old BF-CBC is deprecated. Use AES-GCM on both sides.
• Permission denied on /dev/net/tun inside a container: you need --cap-add=NET_ADMIN --device=/dev/net/tun.
• MTU problems (pages load slowly): a hop in the path has MTU < 1500. tun-mtu 1400 mssfix 1360 fixes this.
• CRL has expired: CRL files have an expiry date. If clients cannot connect after months of uptime, run easyrsa gen-crl again and reload openvpn.
• High CPU on server: many clients share one UDP socket, and OpenVPN is single-threaded. Fix with 2.6+ --server-mt or multiple processes on different ports behind a load balancer.