Storing Terraform State Remotely: Benefits and Best Practices

Quick answer: Storing Terraform state remotely means configuring a backend block so the terraform.tfstate file lives in shared, durable storage such as Amazon S3, Azure Blob Storage, Google Cloud Storage or HCP Terraform instead of on one laptop. Remote state gives your whole team and your CI pipeline a single source of truth, adds locking so two applies cannot collide, and lets you encrypt, version and audit a file that contains secrets.

Local state works until the second engineer joins the project. From that moment on, remote state is not an optimisation but a requirement. This guide compares the major backends, walks through complete, production-ready configurations for AWS, Azure and Google Cloud, shows how to migrate existing state safely, explains how to share outputs between projects, and lists the mistakes that leak secrets or corrupt state.

Why local state breaks down

With the default local backend, state sits in your working directory. That causes four problems the moment infrastructure is shared:

• No single source of truth. Your colleague’s copy and yours diverge after the first independent apply, and Terraform starts proposing to recreate each other’s resources.
• No locking. Two simultaneous applies both write state; one set of changes is silently lost.
• No durability. A lost laptop or a careless rm -rf deletes the only record of what Terraform manages.
• Secrets on disk. State stores passwords and keys in plain text, so a laptop is the worst possible place for it.

If you are still unclear on what the state file actually holds, read Understanding Terraform State: What It Is and Why It Matters first.

Choosing a remote backend

Terraform ships several backends. Pick the one that matches where your infrastructure already lives; introducing a second cloud just for state adds credentials and failure modes.

Note that the artifactory, etcd, manta and swift backends were removed in Terraform 1.3, so older tutorials that mention them no longer apply.

Configuring an S3 backend the right way

The classic example uses just a bucket, key and region. A production configuration adds encryption, locking and a key path that separates projects and environments:

terraform {
  required_version = ">= 1.10.0"

  backend "s3" {
    bucket       = "tkh-terraform-state-123456789012"
    key          = "payments/prod/terraform.tfstate"
    region       = "ap-south-1"
    encrypt      = true
    kms_key_id   = "alias/terraform-state"
    use_lockfile = true   # native S3 locking; no DynamoDB table needed
  }
}

Before Terraform 1.10, locking required a DynamoDB table referenced with dynamodb_table = "terraform-locks" and a partition key named LockID. That still works, but use_lockfile is simpler and the DynamoDB option is being phased out. Whichever you choose, create the bucket itself outside this configuration (or in a tiny bootstrap project with local state), enable bucket versioning, and block all public access.

Keep credentials out of the block. The S3 backend uses the same AWS credential chain as the provider: environment variables, a named profile or an IAM role attached to your CI runner. A role with s3:GetObject, s3:PutObject, s3:DeleteObject on the key path and s3:ListBucket on the bucket is sufficient.

Azure and Google Cloud equivalents

The pattern is identical on other clouds; only the argument names change.

# Azure Blob Storage
terraform {
  backend "azurerm" {
    resource_group_name  = "rg-terraform-state"
    storage_account_name = "tkhtfstateprod"
    container_name       = "tfstate"
    key                  = "payments/prod.tfstate"
    use_azuread_auth     = true   # RBAC instead of storage account keys
  }
}

# Google Cloud Storage
terraform {
  backend "gcs" {
    bucket = "tkh-terraform-state"
    prefix = "payments/prod"
  }
}

Azure Blob provides locking through blob leases automatically, and GCS locks through a .tflock object next to the state. Neither needs an extra service. On Azure, prefer use_azuread_auth over shared access keys so that access is governed by RBAC and auditable in Entra ID. On GCS, enable Object Versioning on the bucket for the same reasons you enable S3 versioning.

Migrating existing state and using partial configuration

To move a project from local to remote, add the backend block and run terraform init. Terraform detects the change and asks:

$ terraform init
Initializing the backend...
Do you want to copy existing state to the new backend?
  Enter a value: yes

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Run terraform plan afterwards; it should report no changes. Only then delete the local terraform.tfstate.

Backend blocks cannot use variables (unlike almost everything else in Terraform, as Using Variables and Expressions in Terraform explains), which is inconvenient when one codebase serves several environments. The fix is partial configuration: leave environment-specific values out of the block and supply them at init time.

# backend.hcl (one per environment, committed to Git)
# bucket = "tkh-terraform-state-123456789012"
# key    = "payments/staging/terraform.tfstate"
# region = "ap-south-1"

terraform init -backend-config=environments/staging/backend.hcl

Switching environments means re-running init with a different file (add -reconfigure if you are not migrating). Many teams instead keep one directory per environment, each with its own full backend block, which avoids the risk of applying staging code against production state.

Sharing outputs between projects

Remote state also lets one root module read another’s outputs. A platform team’s network project exposes vpc_id and private_subnet_ids; the application team consumes them:

data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "tkh-terraform-state-123456789012"
    key    = "platform/network/prod/terraform.tfstate"
    region = "ap-south-1"
  }
}

resource "aws_lb" "app" {
  name    = "payments-prod"
  subnets = data.terraform_remote_state.network.outputs.private_subnet_ids
}

Be aware that this grants read access to the entire state file, secrets included. Where that is unacceptable, publish the needed values to SSM Parameter Store or a tagged resource lookup instead, and read those with ordinary data sources.

Seven remote state mistakes to avoid

1. Reusing one key for several root modules. They will fight over the same state. Use a clear path convention such as <project>/<environment>/terraform.tfstate.
2. Skipping versioning on the bucket. Without it, a corrupted write is permanent. Versioning costs almost nothing.
3. Leaving the state bucket readable by everyone with console access. Scope IAM to the teams that run applies, and enable access logging.
4. Hard-coding access keys in the backend block. They end up in Git. Rely on the credential chain or OIDC in CI.
5. Forgetting locking. Set use_lockfile = true (or a DynamoDB table) on S3; Azure, GCS and HCP lock automatically.
6. Creating the state bucket inside the project that uses it. You cannot destroy that project cleanly. Bootstrap the bucket separately.
7. Running init without -reconfigure or -migrate-state when changing backends. Read the prompt carefully; choosing wrongly can point production code at an empty state.

Frequently asked questions

Is remote state encrypted?

At rest, yes, if you enable it: encrypt = true on S3, default Storage Service Encryption on Azure, Google-managed keys on GCS. In transit all backends use TLS. Encryption does not remove the need for strict access control, because anyone who can read the object can read the secrets inside.

Can I use one bucket for all environments?

Yes, with different key paths, and this is common. For strict isolation some organisations use one bucket per AWS account so that production state never shares an IAM boundary with development.

What happens if the backend is unreachable?

plan and apply fail immediately with a connection or permission error; Terraform never falls back to local state silently. Fix access and retry.

Should I use HCP Terraform instead of S3?

HCP Terraform adds run history, policy checks, variable storage and a UI, and its free tier covers small teams. S3 is cheaper and simpler if you already have a mature CI pipeline. Both are production-grade.

Key takeaways

• Remote state is mandatory as soon as more than one person or a CI system touches the infrastructure.
• Choose the backend of the cloud you already run on; enable encryption, versioning and locking on day one.
• Keep credentials out of the backend block and use partial configuration or per-environment directories for multiple environments.
• One state key per root module, a separately bootstrapped bucket, and tightly scoped IAM.

Want hands-on practice setting up S3 and Azure backends, locking and CI pipelines on real cloud accounts? Our DevOps course covers Terraform state management end to end with mentor support and placement assistance. For video walkthroughs of each backend, visit our YouTube channel.