Process substitution: <(cmd) and >(cmd)

Why

A command often wants a file as input, but what you have is a command that produces the data. The classic example: diff compares two files, and you want to compare the output of two commands.

Without process substitution, you write to temporary files:

ls /etc | sort > /tmp/a.txt

ls /usr | sort > /tmp/b.txt

diff /tmp/a.txt /tmp/b.txt

rm /tmp/a.txt /tmp/b.txt    # don't forget to clean up

With process substitution, one line:

diff <(ls /etc | sort) <(ls /usr | sort)

How it works

Under the hood, <(cmd) is a /dev/fd/N (where N is some FD, usually 63 and up). Bash:

1. Runs cmd in a subprocess
2. Creates a pipe
3. Substitutes the path /dev/fd/63 into the command line, which reads from that pipe

The command (diff) opens this "file" as an ordinary path and reads the subprocess output. No temporary files are written to disk.

echo <(ls)

cat <(echo hello)

The mirror form: >(cmd)

>(cmd) is the opposite: a pseudo-file for writing that a command reads its stream from:

echo "data" | tee >(wc -c > /tmp/byte-count.txt) >/dev/null

Use it to fan the output of one command out to several handlers in parallel:

some_command | tee >(grep ERROR > errors.log) >(grep WARN > warns.log) > /dev/null

Where it does NOT work

• POSIX /bin/sh: no process substitution. Only bash and zsh have it.
• A pipe into process substitution: the exit code is lost. The command inside <(...) can fail and the script won't notice, even with set -e. If the exit code matters, write to a file and check it explicitly.
• Over ssh: ssh host '<(cmd)' won't work, because it runs on the remote side, which may not be bash.

Idioms you will run into

# Compare two directories by content

diff <(ls -la /etc) <(ls -la /etc.bak)

# Check that a command's output matches a file

diff <(my_program) expected.txt && echo OK

# Send STDOUT and STDERR to DIFFERENT files (without 2>&1)

cmd > >(tee out.log) 2> >(tee err.log >&2)

# Use while read to count matches (NOT through a pipe, or the

# variable is lost in a subshell)

count=0

while read -r line; do

    ((count++))

done < <(grep ERROR app.log)

echo "$count"     # ← the variable survived, because there is NO pipe

That last case is a common trap: grep ... | while read; do ((count++)); done does not work (count stays 0), because the pipe runs while in a subshell. Process substitution fixes this.