Troubleshooting and debugging

Что отвечать

`TF_LOG` is the env variable for Terraform diagnostics during a command. The levels are TRACE > DEBUG > INFO > WARN > ERROR. TRACE dumps every HTTP operation of the provider, millions of lines. DEBUG is a bit quieter and is usually enough to see what Terraform was trying to do. INFO is the standard production level. For long-running commands, `TF_LOG_PATH=tf.log` writes to a file instead of flooding the terminal. A handy trick: `TF_LOG_PROVIDER=DEBUG` separate from core shows only the provider logs without the graph internals.

Что хотят услышать

A senior should: - separate TF_LOG (everything) from TF_LOG_PROVIDER (provider only); often you only need the second - name `TF_LOG_PATH` for file output; the terminal log loses its scrollback on large states - say that TRACE gives raw HTTP requests and responses to the provider, which is invaluable for diagnosing auth or a timeout - mention that secrets can settle into the logs (Bearer tokens in Authorization headers), so do not share the files without cleaning them

Подводные камни

• ✗ Running `TF_LOG=TRACE plan` without `TF_LOG_PATH`. The terminal floods, and scrolling up to the useful part takes forever
• ✗ Sharing tf.log in Slack without editing it. It has Authorization headers and sometimes excerpts of state
• ✗ Using TF_LOG=INFO when DEBUG is what you need. INFO shows almost nothing, and turning it up costs you nothing

Follow-up

• ? How does `TF_LOG_PROVIDER` differ from plain `TF_LOG`?
• ? Which log levels make sense to mix?
• ? What can you scrub from a TF_LOG file before sharing it?

Что отвечать

A cycle is a circular dependency in the resource graph. First: `terraform graph | dot -Tsvg > g.svg` shows where it closed on itself. The usual cause is an extra `depends_on` where Terraform already sees an implicit dependency through an attribute, or a `replace_triggered_by` that points back. The fix: drop the duplicate depends_on, break the loop through a local (`local.shared = aws_x.y.attr`, with both resources referencing the local), or move it into a data source if the resource is someone else's.

Что хотят услышать

A senior should: - name `terraform graph | dot` as the first diagnostic tool - give the concrete sources of a cycle: an extra depends_on (the most common), replace_triggered_by, a loop through a child module (`module.A` depends on `module.B` which depends on `module.A`) - explain breaking the loop through a local: if both resources reference one value through locals, the graph comes apart - mention that a cycle is caught only at `plan`, not statically

Подводные камни

• ✗ Deleting an explicit depends_on without checking that the implicit one is really there. The apply fails at runtime
• ✗ Running `terraform graph` without `dot` (a graph tool). The raw output is impossible to read
• ✗ Trying to work around a cycle with `-target`. That treats the symptom, not the cause

Follow-up

• ? How does `terraform graph` help you see the source of a cycle?
• ? How does breaking the loop through a local differ from a data source?
• ? Which HCL constructs most often create a cycle?

Что отвечать

`.terraform.lock.hcl` pins provider versions and their checksums. The error means what is in the lockfile does not match what Terraform found during `terraform init`. The scenarios: (1) someone bumped a version constraint in HCL but did not update the lockfile (you need `init -upgrade`); (2) the lockfile holds a checksum for one platform (linux_amd64) and you are on another (darwin_arm64), so you need `providers lock -platform=darwin_arm64`; (3) the provider published a new artifact for the same version, and then you have to update the checksum by hand.

Что хотят услышать

A senior should: - explain the lockfile's role: reproducible provider versions across runs and machines, and protection against tampered artifacts through the checksum - name `terraform providers lock -platform=X -platform=Y` for multi-platform CI (macOS dev plus linux CI) - say that committing the lockfile is mandatory; a repo without `.terraform.lock.hcl` gives you "works on my machine, not on yours" - mention that `init -upgrade` recomputes the lockfile, so use it deliberately, not by default

Подводные камни

• ✗ Not committing the lockfile. There is no reproducibility, and provider versions drift over a year
• ✗ Running `init -upgrade` by default in a pipeline. You get surprise behavior on a minor provider bump
• ✗ Building the lockfile for linux_amd64 only. Developers on Macs fail at init

Follow-up

• ? Why run `terraform providers lock -platform=darwin_arm64`?
• ? What happens if you commit a lockfile with linux checksums only?
• ? How does `init -upgrade` differ from a plain init?

Что отвечать

`terraform console` is a REPL for checking expressions interactively against the current state. It helps when something in HCL is odd and you do not see how a function behaves. You run `terraform console`, type `{ for k, v in aws_subnet.public : k => v.cidr_block }`, and see the result right away, with no apply. Also `length(var.list)`, `try(local.maybe_missing, "default")`, checking type coercion. It only works with an already initialized root, and it reads state.

Что хотят услышать

The candidate should: - name console as the way to check an expression quickly, without editing HCL or running `terraform plan` - say that console reads the real state, so computed attribute values are already there - mention multi-line input through brackets or a heredoc for complex expressions - note that console will not start without `terraform init`; it needs the providers to know the resource schema

Подводные камни

• ✗ Thinking console can evaluate an expression that references a resource not yet in state. It cannot
• ✗ Trying to do `aws_instance.demo = ...` in console. It is a REPL for expressions, not for updates
• ✗ Not exiting console and losing the lock. console does not hold a lock, but in a long session the state can go stale

Follow-up

• ? Can console reach the provider API?
• ? How do you test `try()` or `coalesce()` through console?
• ? Why do you need `terraform init` before console?

Что отвечать

`+` is create. `-` is destroy. `~` is update in place (the attribute changes, the resource stays). `-/+` is destroy then create (a replace, a full recreation, usually an immutable attribute). `+/-` is create then destroy (the same, but with `create_before_destroy`). `<=` is read (a data source). `# (will be unchanged)` is a no-op. The thing that matters most in review is telling `~` (in place, safe) from `-/+` (a replace, possible downtime). A replace driven by `-target` is always suspicious.

Что хотят услышать

A senior should: - tell an in-place update (`~`) from a replace (`-/+`); a replace is destroy the old plus create the new, which often means downtime - say that `+/-` is a replace through `create_before_destroy`, where the new one is created before the old is destroyed - mention that Terraform gives the detail on why a resource is replaced: "(forces replacement)" next to the attribute - name `terraform show -json plan.tfplan | jq` for an automated plan check in CI without a visual review

Подводные камни

• ✗ Missing a `-/+` in the plan and applying. Downtime in production
• ✗ Assuming `~` is always safe. On some resources an in-place update also causes downtime (RDS engine_version)
• ✗ Parsing plan output with a regex instead of JSON. The format changes between versions

Follow-up

• ? How is `-/+` fundamentally different from `+/-`?
• ? How do you find the 'forces replacement' reason in the plan output JSON?
• ? When does `~` still cause downtime?

Что отвечать

graph prints the resource DAG in DOT format. It helps in a few cases: a cycle, where you see where it closed on itself; understanding graph density after a refactor, whether you are adding extra edges through depends_on; reviewing large infrastructure, who references whom. In everyday work you do not need it; the plan and the diff show everything you need for an apply. The Rover tool (`im2ag/rover`) builds an interactive web visualizer on top of graph, which is handier for a "here is our infrastructure" conversation with the team.

Что хотят услышать

A senior should: - name the concrete scenarios where graph pays off (a cycle, a refactor review, knowledge transfer) - say that raw DOT is unreadable; always go through `dot -Tsvg` or Rover - mention that graph does not show provider attributes, only the dependencies between resources and data sources - note that for large infrastructure graph becomes a monster: 5000 nodes in an svg, and you cannot make anything out, while rover with filters is more effective

Подводные камни

• ✗ Running `terraform graph` without `dot` or Rover. It is impossible to read
• ✗ Using graph for CI checks ('did a cycle appear'). The plan phase catches a cycle on its own
• ✗ Assuming graph shows the real infrastructure topology. It shows Terraform's management graph, not the network

Follow-up

• ? Why is Rover better than flat dot output?
• ? What does graph show for modules?
• ? Which edges in the graph can you safely ignore?