Core concepts
- qdisc (queueing discipline): the queue rule on an interface.
Every interface always has a qdisc. The default is
pfifo_fastorfq_codel. - class: a hierarchy inside a qdisc (for splitting bandwidth between traffic types)
- filter: what goes into which class (matched by port, IP, or mark)
Viewing queues
bash
tc qdisc show # qdisc on all interfaces
tc qdisc show dev eth0
tc -s qdisc show dev eth0 # -s: with statistics (bytes, packets, dropped)
tc class show dev eth0
tc filter show dev eth0
netem: simulating a bad network
The most common use case for CI and chaos testing:
bash
sudo tc qdisc add dev eth0 root netem delay 200ms
sudo tc qdisc add dev eth0 root netem delay 100ms 20ms # +-20ms jitter
sudo tc qdisc add dev eth0 root netem loss 5%
sudo tc qdisc add dev eth0 root netem corrupt 1%
sudo tc qdisc add dev eth0 root netem duplicate 1%
sudo tc qdisc add dev eth0 root netem reorder 25% 50%
# Combine multiple effects at once:
sudo tc qdisc replace dev eth0 root netem delay 50ms loss 1%
# Remove:
sudo tc qdisc del dev eth0 root
To verify the effect, ping through the interface and watch for retransmits
in cmd-ss -ti.
tbf: bandwidth limiting
Token Bucket Filter is a straightforward rate limiter:
bash
sudo tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 400ms
▸1 Mbit/s outgoing, queue up to 400ms
htb: hierarchical bandwidth sharing
bash
# 100Mbit on the interface, split: 70 for class 1:10, 30 for 1:20
sudo tc qdisc add dev eth0 root handle 1: htb default 20
sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit
sudo tc class add dev eth0 parent 1:1 classid 1:10 htb rate 70mbit ceil 100mbit
sudo tc class add dev eth0 parent 1:1 classid 1:20 htb rate 30mbit ceil 100mbit
# Route traffic: SSH (port 22) goes to the premium class
sudo tc filter add dev eth0 parent 1: protocol ip prio 1 \
u32 match ip dport 22 0xffff flowid 1:10
Applying to loopback (for local tests)
netem on lo is the fastest way to run chaos tests locally, without building a network topology:
bash
sudo tc qdisc add dev lo root netem delay 200ms
ping -c 3 127.0.0.1
▸RTT is now ~200ms
sudo tc qdisc del dev lo root
Capability
All tc add/del operations require CAP_NET_ADMIN. In containers, add
--cap-add=NET_ADMIN.