When to use
- A process is stuck. Which syscall is it blocked on?
- A file won't open. Which exact path, and what error?
- A network client is slow. Where is it connecting, and how long does it wait?
- Comparing two binaries. Why is one faster?
Under the hood: ptrace(2). On many systems you need the SYS_PTRACE
capability or ptrace_scope=0 in sysctl.
Basic forms
strace ls /tmp # run ls under strace
strace -p 1234 # attach to an existing [[process-and-pid]]
strace -f -p 1234 # -f: follow child processes
strace -c ls /tmp # -c: summary only (count + total time per syscall)
Filters
strace -e trace=openat ls # only openat()
strace -e trace=network curl URL # network only: connect, sendto, recvfrom...
strace -e trace=file ls # all file-related: open, stat, access, read...
strace -e trace=!futex,clock_gettime ls # ! = EXCLUDE noisy calls
Useful flags
-t/-tt/-ttt- timestamps (increasing precision)-T- duration of each syscall in brackets after the return value-y- resolve file descriptors to file names-yy- same, plus socket types-s 256- show longer strings in arguments (default is 32)-o file.log- write output to a file instead of stderr-f- follow child processes (without-f, strace tracks only the parent)
Typical output
openat(AT_FDCWD, "/etc/passwd", O_RDONLY) = 3
read(3, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1234
close(3) = 0
When a syscall fails, you see it immediately:
openat(AT_FDCWD, "/no-such", O_RDONLY) = -1 ENOENT (No such file or directory)
strace -c for performance
The summary shows where time is spent:
strace -c -p $(pgrep -x app) -- sleep 10
▸table: syscall / % time / total / calls / errors
▸80% of time in epoll_wait means the app is waiting on I/O, that is normal
▸80% in futex means locks are contending, look for contention
Limitations
- Tracing slows the process by 10-100x. Do not run it in production for long.
- It does not show kernel-level activity (what happens inside the kernel between
sys_enter and sys_exit). Use
bpftrace/perf tracefor that. - It cannot see processes in a different PID namespace without the right privileges.