Remote state in S3: bucket, DynamoDB lock, encryption

Why remote state

Local state lives next to your HCL. That works when you are the only person on the project. As soon as a team is involved, problems start.

• Someone runs apply from their laptop. State is on their disk. A second colleague has a different state. They apply different plans to the same infrastructure. Chaos.
• The laptop dies or gets wiped, state is lost. Recover by running terraform import on each resource by hand.
• CI/CD has no state at all. Every run thinks the infrastructure has not been created.

A remote backend is a shared state, accessible to everyone (within their permissions), with locks that prevent parallel applies. S3 + DynamoDB is the most common combination in AWS projects.

Architecture

+----------+      +-----------+      +-----------+

| dev      |      | S3 bucket |      | DynamoDB  |

| laptop A | ───→ | tfstate   | ←─── | lock      |

+----------+      +-----------+      +-----------+

                     ↑                    ↑

                     │                    │

+----------+    plus │            check   │ before

| dev      |    write                     │ write

| laptop B | ──────────────────────────────┘

+----------+

• S3 stores the terraform.tfstate file. S3 versioning must be enabled; it protects against accidental overwrites.
• DynamoDB stores the lock record. Before a write, Terraform creates the lock; after apply it deletes it. A parallel attempt sees the lock and fails with a clear error.
• Encryption. S3 server-side encryption is on by default. You can also use AWS KMS with your own key for compliance requirements.

Minimal backend configuration

You can create the resources (S3 + DynamoDB) with a separate "bootstrap" root module or manually through the AWS CLI. Do it once, then reference them:

terraform {

  backend "s3" {

    bucket         = "myorg-terraform-state"

    key            = "billing/prod/terraform.tfstate"

    region         = "us-east-1"

    dynamodb_table = "myorg-terraform-locks"

    encrypt        = true

  }

}

Key fields:

Backend values cannot be interpolated

# DOES NOT WORK

terraform {

  backend "s3" {

    bucket = var.state_bucket   # ERROR: variables are not allowed here

  }

}

The backend is configured before Terraform reads variables. If you need to parameterize across environments, use partial backend config:

# terraform { backend "s3" {} } , empty block

Then pass values at init time:

terraform init -backend-config="bucket=myorg-terraform-state" \

               -backend-config="key=billing/prod/terraform.tfstate" \

               -backend-config="region=us-east-1" \

               -backend-config="dynamodb_table=myorg-terraform-locks"

Or use -backend-config=prod.hcl with a file. This is the standard technique for a multi-environment layout.

Bootstrap: creating the bucket and table

Chicken-and-egg problem: to store state in S3, the S3 bucket must already exist. There are two solutions.

Option 1: manually via AWS CLI

aws s3api create-bucket --bucket myorg-terraform-state --region us-east-1

aws s3api put-bucket-versioning --bucket myorg-terraform-state \

    --versioning-configuration Status=Enabled

aws s3api put-bucket-encryption --bucket myorg-terraform-state \

    --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'

aws dynamodb create-table --table-name myorg-terraform-locks \

    --attribute-definitions AttributeName=LockID,AttributeType=S \

    --key-schema AttributeName=LockID,KeyType=HASH \

    --billing-mode PAY_PER_REQUEST

Done once per organization. The state of these resources is not managed by Terraform; they are created manually, documented, and left alone.

Option 2: a separate bootstrap module with local state

One root module creates the bucket and table using aws_s3_bucket and aws_dynamodb_table. Its state lives locally (or in git as bootstrap.tfstate, for audit purposes). All other root modules use the S3 backend.

This approach is cleaner (everything is described in HCL), but you need to store and protect the bootstrap state somewhere. It often goes in a separate private repository.

Migrating from local to S3

You have a terraform.tfstate file and want to move it to S3.

1. Add a terraform { backend "s3" { ... } } block to your HCL.

2. Run:

   bashcopy

   terraform init -migrate-state

   Terraform will ask: "Found local state and a new remote backend, migrate it?" Answer yes.

3. The local terraform.tfstate remains on disk as terraform.tfstate.backup. Do not delete it immediately; give it a week for verification.

Reverse migration (from S3 to local): use the same init -migrate-state, but remove the backend block from HCL first. Terraform will ask for confirmation again.

Locking in action

When you run apply, Terraform writes to DynamoDB:

LockID:    myorg-terraform-state/billing/prod/terraform.tfstate-md5

Operation: OperationTypeApply

Who:       alice@laptop

Created:   2026-05-26 14:00:00 UTC

Info:

A parallel apply will see the lock and fail:

Error: Error acquiring the state lock

Lock Info:

  ID:        ...

  Operation: OperationTypeApply

  Who:       alice@laptop

  Created:   2026-05-26 14:00:00 UTC

See tf-common-errors for details on Error acquiring the state lock.

force-unlock

If a process died (Ctrl+C, crash), the lock remains. Use terraform force-unlock <ID>. Dangerous: if the process is still running, this allows a second apply and corrupts state. Use only when you are certain the process is gone.

Common pitfalls

• Without bucket versioning, a lost state means a lost infrastructure. An accidental terraform state rm at the root followed by apply recreates everything. S3 versioning lets you roll back to a previous state version. Make it mandatory.

• One S3 bucket per organization. Do not create a bucket per project. Permissions become harder to manage and billing gets confusing. Use one bucket with different key paths. An IAM policy controls who can write to which key.

• encrypt = true is the minimum. For compliance, use SSE-KMS with a customer-managed key so that access to state requires a KMS permission separate from the S3 permission. Two layers.

• The DynamoDB table must be in the same region as the bucket. Cross-region calls slow down locking. Best practice is to put both in your organization's primary region.

• LocalStack emulates the S3 backend, but not perfectly. In our tutorials we use LocalStack, which works for learning and integration tests. In production you will use real S3 + DynamoDB. Behavior in edge cases (large state, slow networks, eventual consistency) differs.

• key cannot be interpolated at apply time. It is part of the backend config and is resolved at init. To switch between prod and dev state, use two separate init configurations or Terraform workspaces (see tf-workspace).

• Backups on top of S3 versioning are still worth having. Versioning protects against accidental overwrites. It does not protect against bucket deletion. For serious projects, set up cross-account backup via S3 replication or AWS Backup.