Why NFS
Network File System is the standard way to "mount someone else's disk over the network" on Unix. It is transparent: open/read/write/close work like on a local FS, and everything is cached by the page cache. It is used for:
- Shared home directories in corporate networks
- Storage for VM images (KVM, Proxmox)
- ML datasets shared between GPU nodes
- Backup targets
- Kubernetes Persistent Volumes (NFS-CSI driver)
Alternatives: SMB/CIFS (native to Windows), [[cmd-rsync|rsync]] (for periodic sync, not a live mount), CephFS/GlusterFS (distributed), object storage (S3, for non-POSIX).
NFSv3 vs NFSv4
| Trait | v3 (1995) | v4.0 (2003) / v4.1 (2010) |
|---|---|---|
| State | stateless | stateful |
| Transport | UDP or TCP | TCP only |
| Ports | 111 (rpcbind) + dynamic | 2049 single port |
| Lock manager | rpc.lockd (separate) | built in |
| Auth | sys (uid/gid from RPC) | + Kerberos (krb5/krb5i/krb5p) |
| Delegations | no | yes (read/write delegation) |
| pNFS | no | yes in v4.1 (parallel NFS) |
| ACL | POSIX via protocol-extension | NFSv4 ACL (Windows-style) |
| Through NAT/firewall | hard (dynamic ports) | simple (one port) |
NFSv3 is still alive in legacy and embedded. For new code use NFSv4.1+.
Server: /etc/exports
# Install
apt install nfs-kernel-server # Debian/Ubuntu
dnf install nfs-utils # RHEL
# Create a share
mkdir -p /srv/nfs/data
chown nobody:nogroup /srv/nfs/data
/etc/exports:
/srv/nfs/data 10.0.0.0/24(rw,sync,no_subtree_check,root_squash)
/srv/nfs/backup 10.0.0.5(ro,sync,no_subtree_check) 10.0.0.6(rw,sync,no_subtree_check)
/srv/nfs/krb5 *(rw,sync,sec=krb5p,no_subtree_check)
Apply:
exportfs -ra # re-export everything
exportfs -v # show active exports
systemctl enable --now nfs-server
Main exports options
| Option | What |
|---|---|
rw / ro | access for the subnet |
sync (default) | reply to the client only after fsync |
async | reply right away, fsync later, faster, but data is lost if the server crashes |
root_squash (default) | the client's root maps to nobody, a protection |
no_root_squash | the client's root stays root, very dangerous |
all_squash | all users map to nobody (for a public share) |
anonuid=, anongid= | what to map squashed users to |
subtree_check / no_subtree_check | check that the path is inside the export; no_subtree_check is faster, default on modern systems |
| `sec=sys | krb5 |
nohide | for nested mounts on the server |
crossmnt | cross mount points while walking |
Client: mount
apt install nfs-common # Debian
mount -t nfs server:/srv/nfs/data /mnt/data
mount -t nfs4 server:/srv/nfs/data /mnt/data
In [[mount-and-fstab|fstab]]:
server:/srv/nfs/data /mnt/data nfs4 rw,_netdev,vers=4.2,hard,timeo=600,retrans=3,nofail 0 0
Main mount options:
| Option | What |
|---|---|
vers=4.2 | force the version (default is auto-negotiate) |
hard (default) | on a lost connection, block I/O until it comes back |
soft | return an I/O error after timeo*retrans ms, dangerous, may corrupt data |
timeo=N | timeout in deciseconds (600 = 60 sec) |
retrans=N | how many retries before declaring the "server dead" |
_netdev | bring up the network first (for systemd) |
nolock | without NLM (for read-only mounts) |
nfsvers=3,proto=tcp,port=2049 | for hard pinning of v3 |
noatime | as everywhere, turn off atime |
nconnect=N | (5.3+) N TCP connections to the server, boosts throughput |
hard vs soft: always use hard. soft gives failover in theory,
but in practice it corrupts files on short network outages.
Caching and consistency
NFS does not guarantee strict "wrote on one node, visible on another
immediately". The model is close-to-open consistency: on close() the
client flushes, and on open() another client checks mtime via
GETATTR and invalidates its cache.
This means NFS is not suitable for a shared DB (different clients will not see edits). For a shared home, build output, or ML dataset, it is fine.
Kerberos: sec=krb5p
Without Kerberos the NFS client states its own UID through RPC AUTH_SYS, with trust at the level of "we use the same UIDs on every machine". In multi-user enterprise environments that is weak.
With sec=krb5p, every operation is authenticated by the user's Kerberos
ticket, and the payload is encrypted. It requires:
- A KDC (kerberos)
- The service principal
nfs/server.example.com@REALMin the server keytab idmapd.confto mapuser@REALMto a UID on the clients
It is complex, but the only production-grade option for shared homes.
NFSv3 vs NFSv4 firewall
v3:
- 111 (portmap/rpcbind) TCP/UDP
- 2049 (nfs) TCP/UDP
- mountd, statd, lockd, rquotad on dynamic ports!
- Fix: pin the ports in
/etc/nfs.conf
v4:
- 2049 TCP only, everything through one port, including the lock manager
- Works through NAT without pain
This is one of the main reasons to migrate to v4.
When something goes wrong
mount.nfs: Connection refused: nfs-server is not running or is not listening on 2049 (ss -tlnp | grep 2049).ls /mnt/datahangs forever: ahardmount with the server unreachable.umount -f -l /mnt/data(lazy umount).Stale file handle: the file was deleted on the server while the client holds an fd.umount && mountfixes it. Less frequent on v4.1+.- Other users' files show as
nobody:nogroup: no idmapd mapping. On NFSv4, make surenfs-idmapdis running and thatDomain =matches in/etc/idmapd.confon both ends. - Performance is slow on large files:
wsize=1048576,rsize=1048576,nconnect=4(5.3+), and check jumbo frames on the wire. Permission denieddespite mode 777:root_squashmaps root to nobody. Create files as a regular user, or useno_root_squash(only on a trusted network!).- Lockd does not work between hosts: on v3 you need
rpc.statdplus portmap; on v4 it is built in. Check that the firewall is not blocking the statd port on v3.
Alternatives
- SMB/CIFS (Samba): the Windows standard, also works on Linux
- CephFS: distributed, replicated, for scale
- GlusterFS: same idea, deprecated after IBM absorbed Red Hat
- sshfs (FUSE): a tunnel over SSH, for one-off cases
- S3/MinIO: for non-POSIX object storage