how/state

Drift: when state and reality diverge

Someone changed a tag in the AWS Console, and Terraform suddenly shows changes. What drift is, why it happens, and three ways to live with it.

Drift is when the real state of a resource in the cloud diverges from what Terraform holds in its state.

It happens for several reasons:

someone with permissions fixed something "just for a second" in the AWS Console.
auto-scaling adjusted a parameter on its own.
another tool (kubectl, ansible, a script) changed the resource in parallel.
the AWS provider updated an attribute that Terraform does not control.

Press ▶ to follow the life cycle of a single drift: how it appeared, how Terraform saw it, and three ways to resolve it.

step 1/6·00 · after apply: the invariant holds

§ steps

Right after a successful terraform apply, all three sources of truth match: the HCL describes a bucket with the tag Owner = "student", the state knows about this tag, and AWS has exactly that bucket.

This is the base state of Terraform. Any repeat plan is clean.

recap

Three strategies for dealing with drift:

Roll back with apply. terraform apply brings the cloud back to the HCL. Use this when the HCL is the truth and the cloud is wrong.
Pick it up into the HCL. If the change in the cloud was correct, you edit main.tf so it reflects the new truth, then run apply (or refresh plus a manual edit of the state block).
Ignore it with [[tf-resource-lifecycle|lifecycle.ignore_changes]]. If a field naturally drifts (auto-scaling changes desired_count, the cloud sets a timestamp itself), you mark it as ignored, and Terraform stops treating it as a divergence.

Which strategy and when:

a manual fix in the cloud to put out an incident → strategy 2 (pick it up into the HCL during the post-mortem so the invariant holds again).
auto-scaling, tags applied by the cloud → strategy 3.
everything else → strategy 1, through CI and a pull request.

A regular drift check in CI (see tf-drift-detection) is mandatory in production. Drift that no one sees turns into an incident.