What an fd is
When a process-and-pid opens something (a file, socket, or pipe), the
kernel returns a small integer, the file descriptor. From then on the
process reaches the resource by that number: read(fd, ...), write(fd, ...),
close(fd).
An fd is an index into the process table: /proc/<pid>/fd/N points at the target.
The three "magic" fds
Every process that starts gets three fds by default:
| fd | name | default |
|---|---|---|
| 0 | stdin | keyboard (terminal) |
| 1 | stdout | screen |
| 2 | stderr | screen (a separate stream!) |
This is the Unix convention: a program reads from 0, writes its result to 1, and sends errors to 2. The shell can redirect each stream on its own.
Redirections in bash
cmd > out.log # stdout → file (= 1>out.log)
cmd 2> err.log # stderr → file
cmd > all.log 2>&1 # stdout to the file FIRST, then stderr to the same place
cmd > out.log 2> err.log # split them into separate files
cmd &> all.log # bash shortcut: both stdout and stderr into one file
cmd < input.txt # stdin ← file
cmd << EOF ... EOF # heredoc: stdin ← inline text
cmd <<< 'one line' # here-string: stdin ← a single line
IMPORTANT: 2>&1 means "make fd 2 a copy of fd 1". Order matters.
cmd >file 2>&1 is correct (stdout goes to the file, then stderr goes to
the same file), while cmd 2>&1 >file is not (stderr went to wherever
stdout WAS, the terminal).
/dev/null and /dev/stderr
cmd > /dev/null # throw away stdout
cmd 2> /dev/null # throw away stderr (the usual "do not show errors")
cmd > /dev/null 2>&1 # throw away everything
/dev/null is a special file: anything written to it is discarded, and a read
returns EOF.
Pipe = an fd between processes
ls | grep '.txt'
The shell creates a pipe (two linked fds) and forks both processes. For
ls, stdout is replaced with the write end of the pipe; for grep, stdin is
replaced with the read end. You get a stream between them without any files.
/proc/<pid>/fd: what is open right now
ls -l /proc/$$/fd/ # fds of the current shell (symlinks to targets)
ls -l /proc/$$/fd/0 # usually → /dev/pts/X (the terminal)
ls -l /proc/<pid>/fd/ # any process (needs permissions)
This is the same thing cmd-lsof -p <pid> shows.
Limits
How many fds one process can hold at once:
ulimit -n # current soft limit (usually 1024 or 65535)
cat /proc/sys/fs/file-max # global limit across ALL processes
"Too many open files" means you hit the limit. Raise it with ulimit -n in the
shell or LimitNOFILE= in a systemd unit.
Duplicating an fd: dup2()
From C code, dup2(src, dst) copies src into dst. This is the basis of
redirection: the shell runs dup2(open("file"), 1) to redirect a process's
stdout before the exec.