Backends in Terraform: where state lives

What a backend is

A backend is the address where terraform stores the state file and where it takes a lock during apply. By default the backend is local: terraform.tfstate sits in the same folder as your HCL, with no locks beyond a file flag.

The backend is configured in the terraform block:

terraform {

  backend "local" {

    path = "terraform.tfstate"

  }

}

The local backend is the default, so you do not have to declare it explicitly.

Why move away from local

The local backend works as long as:

• the project has one person;
• the state is not critical;
• there is no CI/CD that also wants to apply.

As soon as a second person or CI joins the project, you need:

• a shared store, so everyone sees one state instead of each holding their own local copy;
• a lock, so that two parallel runs of apply do not corrupt the state file (one writes, the second writes over it, and you get garbage);
• versioning/backup, in case someone accidentally damages the state.

Remote backends cover these requirements.

Which remote backends exist

The course does not configure them (that is intermediate material), but knowing they exist is useful. They are all declared in a backend block with the matching name.

S3 (the most common in AWS projects)

terraform {

  backend "s3" {

    bucket         = "myorg-tfstate"

    key            = "projects/web/terraform.tfstate"

    region         = "us-east-1"

    encrypt        = true

    dynamodb_table = "tf-locks"        # for locks

  }

}

The bucket stores the state, and the DynamoDB table holds the lock. With Terraform 1.10+ you can keep locks directly in S3 (through S3 conditional writes), and then DynamoDB is not needed.

Google Cloud Storage

terraform {

  backend "gcs" {

    bucket = "myorg-tfstate"

    prefix = "projects/web/"

  }

}

The lock lives in GCS too (through generation-based if-conditions).

HTTP (universal)

terraform {

  backend "http" {

    address        = "https://example.com/state/web"

    lock_address   = "https://example.com/state/web/lock"

    unlock_address = "https://example.com/state/web/lock"

  }

}

Any service that implements the expected REST contract (GET/POST state, LOCK/UNLOCK) works here. Atlantis, GitLab managed state, and most custom solutions are built on it.

Terraform Cloud / Enterprise

Not exactly a backend. It is a separate cloud {} block with workspaces and run history. This is a management platform in which state is just one of the details.

How the backend changes

Once you change backend in HCL, the next init asks:

Initial configuration of the requested backend "s3"

Do you want to copy existing state to the new backend?

  Pre-existing state was found while migrating the previous "local"

  backend to the newly configured "s3" backend. No existing state

  was found in the newly configured "s3" backend. Do you want to

  copy this state to the new "s3" backend?

  Enter "yes" or "no":

• yes tells terraform to move the state from the old location to the new one.
• no leaves it as is; the new backend starts with empty state.

To skip the question, there are flags:

• terraform init -migrate-state says "migrate" explicitly.
• terraform init -reconfigure reinitializes the new backend without moving anything. The old state stays as is, and the new one is empty.

If you do not understand which flag you are pulling, do not rush: you can wipe out a working state.

State locking: what it is and why it matters

A lock means "busy, an apply is running, do not interfere." Without it:

• two engineers run apply at the same time against two different states. Whoever saves last wins the state.
• an apply fails halfway through, and without a lock another one can start over the top and finish the damage.

The local backend takes a lock through a file advisory lock (.tfstate.lock.info). That is enough for a single process, but not for a team.

The S3 backend uses a DynamoDB table or S3 conditional writes. Terraform Cloud has a built-in mechanism. HTTP locks through lock_address.

If another process holds a lock and dies, there is terraform force-unlock <id>. This lever is dangerous, so use it carefully: you can break a live apply that is simply slow.

Pitfalls

• You cannot parameterize backend with variables. Inside a backend "s3" block you cannot write bucket = var.bucket. This is because the backend is read before HCL is fully loaded. The fix is -backend-config flags at init time, or a separate backend.tf in different directories.
• State in the local backend is plain text. Any password, token, or secret_key sits in it as is. In production environments you solve this with a remote backend that encrypts the data.
• Init without a backend is a trap. If you remove the backend block from HCL, terraform decides you have moved to local and asks about migration. Do not confuse "no backend in the code" (== local) with "an empty backend "local" {}".
• force-unlock is a weapon of mass destruction. If the lock is held by a genuinely running apply and you force it off, the second apply will write its state over the top. You can break the state for hours.