Troubleshooting
Broken machines to repair. You get the symptom, not the cause.
In a sandbox lesson you know what to do and you learn to do it. Here you only know what does not work. The container arrives already broken, there are no ready commands, and the system checks your repair against criteria inside your own machine.
Scenarios
- 1nginx will not start
The service does not come up and the address serves someone else's page. Find both causes and bring the site back.
4 criteriaFreeWork on it - 2The disk is full and nothing is using it
df says the disk is full. du walks the same directory and finds almost nothing. Get the free space back.
3 criteriaFreeWork on it - 3The disk is empty and the file will not be created
df shows an empty filesystem, yet no new file can be created: "No space left on device". Work out why and make writing possible again.
3 criteriaFreeWork on it - 4The script is there and refuses to run
The release script is where it should be, its mode is fine, /bin/bash exists, and running it answers required file not found. Through bash it works.
3 criteriaFreeWork on it - 5The service is running and unreachable
The process is alive, its own log says it is listening, and it answers from the machine itself. From outside, nothing gets through.
3 criteriaFreeWork on it - 6The scheduled job says nothing
The developer ran the script by hand and it worked. On schedule it produces nothing and complains about nothing. Find both causes.
3 criteriaFreeWork on it - 7The file's permissions are right and it still cannot be read
The deploy script fails reading its configuration. The file itself is mode 644 and readable by anyone.
4 criteriaFreeWork on it - 8The service shows as running but does not answer
The init script says running and refuses to start. Meanwhile nobody listens on the port. Find out whom it believes.
4 criteriaFreeWork on it - 9nginx answers 502 Bad Gateway
nginx answers, but /api/ returns 502. Both nginx and the backend are running. Find what sits between them and bring the answer back.
4 criteriaFreeWork on it - 10The site answers 403 Forbidden
nginx is running, the configuration passes its check, and the address returns 403 instead of the page. Find out who is forbidden to do what.
5 criteriaFreeWork on it - 11The database is up and the application cannot find it by name
The orders-api health check fails with "Name or service not known". The database listens on the same machine and the network is fine.
4 criteriaFreeWork on it - 12Nobody can log in as deploy
The user exists and its files are in place, yet su and sudo -i both refuse, each with its own message. Bring the service account back without recreating it.
5 criteriaFreeWork on it - 13The connection hangs and times out
The service listens on every address and answers instantly from the machine itself. A request to the machine's address hangs until it times out instead of being refused.
3 criteriaFreeWork on it
How it works
- 01
The machine is already broken
The fault is planted as root before you attach. The briefing names the class of failure and the method, not the fault itself: you find it in the terminal, and the exact answer sits on the last rung of the hint ladder.
- 02
Criteria, not steps
Instead of a list of actions there is a definition of done. The check runs in your own container and tells you which conditions are still unmet.
- 03
Hints one at a time
The hint ladder is closed and opens rung by rung, from where to start to the plain answer. Nothing counts them and nothing is deducted.
Common questions
How is this different from the sandbox lessons?
A lesson gives you a task and the commands for it: you know what to do and you learn to do it quickly and correctly. A scenario gives you a symptom. The order of work is yours to choose, and it differs from person to person.
Do I need an account?
You can read a scenario without one. Booting the container needs a sign-in: the machine runs on our server and its resources are finite. Four scenarios open with a free account, the rest are part of the paid course.
Why does systemctl not work?
The sandbox is a container, not a virtual machine, and systemd is not running in it: the first process is an ordinary shell. That is why systemctl answers "System has not been booted with systemd as init system" and does nothing. There is a replacement: the service command starts and stops services (service nginx start, service nginx status), the service's own tooling reports on its configuration, and the reasons for a failure sit in its log files under /var/log. This is exactly how you work on a service inside a container at your job too.
Can I break something for real?
Inside the container, as much as you like. It is throwaway, it has no route to the internet, and it is deleted when you leave. The restart button rebuilds the machine along with its original fault.