WorksOnMyCluster

What State Is and Why It Must Exist

The four reasons Terraform cannot work by reading the world on every run, what state actually contains, why it holds your secrets in plain text, and the inspection commands that tell you what Terraform believes.

intermediate 18 min lesson hands-on task included

State is Terraform’s record of everything it manages: resource IDs, cached attribute values, dependency ordering, and outputs. It is the product, not a by-product — and it is the single artifact whose loss ruins your day.

Topic 1: Why Terraform Cannot Just Read the World

The obvious question: why not inspect the live infrastructure on every run and compare it to the code? Four reasons, and each is sufficient on its own.

1. Write-once values. A database password can be set but never read back from the API. Without state, Terraform could never detect that you changed it in code, because it has no record of what it set.

2. Ownership. Terraform must not assume everything in your account is its to delete. State is the record of what it created and therefore what it may destroy. Without it, either Terraform deletes things it did not create, or it can never delete anything.

3. Deletion ordering. State stores the dependency graph as it was at creation time, so destroy runs in reverse order. The only alternative would be for Terraform to hard-code the dependency order of every resource type in every provider — unmaintainable, and it would not scale.

4. Performance. For small infrastructures Terraform refreshes every resource on every plan. For large ones that is too slow, so operators use -refresh=false and -target, treating cached state as the record of truth.

Topic 2: What State Contains

terraform show                    # human-readable
terraform show -json | jq         # machine-readable

Inside you will find:

  • Resource IDs — the provider’s identifier for each object.
  • Every attribute Terraform knows, cached from the last refresh.
  • Dependency relationships, so destroy can be ordered.
  • Outputs, including their values.
  • A schema version per resource, used when providers change their internal format.

Default location is terraform.tfstate in the working directory, JSON-formatted. A terraform.tfstate.backup holds the previous version.

Topic 3: State Holds Your Secrets

Database passwords, generated private keys, and any sensitive attribute land in state in plain text. This is not a bug and there is no flag that changes it — Terraform must record what it set in order to detect drift.

The consequences drive nearly every recommendation in this path:

  • Never commit terraform.tfstate to version control. Add it to .gitignore alongside *.tfvars.
  • Encrypt state at rest in the backend.
  • Restrict who can read the backend as tightly as you restrict production access, because they are equivalent.
  • Marking a variable or output sensitive suppresses display only — the value is still in state.

Topic 4: Inspecting State

terraform state list                        # every resource address
terraform state show aws_instance.web       # all attributes of one resource
terraform state pull > backup.tfstate       # download remote state

state list then state show is the diagnostic pair. When something behaves unexpectedly, these tell you what Terraform believes; the plan tells you what it intends to do about it. Reconcile those two before touching anything.

$ terraform state list
aws_instance.web
aws_security_group.web
aws_vpc.main
module.network.aws_subnet.public[0]
module.network.aws_subnet.public[1]

Note the address format — module resources are prefixed with module.<name>., and instances created by count or for_each carry their index or key. Those addresses are what every state manipulation command takes as an argument.

Topic 5: Refresh and the Cached-State Tradeoff

By default, plan refreshes state against reality before diffing. On a large estate that means an API call per resource, which is slow and can hit rate limits.

terraform plan -refresh=false          # trust cached state, skip the API calls
terraform plan -target=module.network  # limit the scope of the operation

Both speed things up considerably. Both mean Terraform will not notice out-of-band changes on that run. That is an acceptable trade for iteration speed on a big project and an unacceptable one before a production apply.

-target in particular is a debugging tool, not a workflow. Routinely targeting means your state is too large and should be split — a topic the project-layout lesson picks up.

Topic 6: Never Hand-Edit State

You will eventually be tempted. The pressure is real: something is wrong, the fix is obviously one line in a JSON file, and the proper command is not obvious.

Use the commands instead — the next lesson covers mv, rm and import. They validate what they write; a text editor does not, and a malformed state file is a much worse problem than the one you started with.

If you genuinely must, terraform state pull first so you have the exact bytes to restore, and make sure backend versioning is enabled.

Try it yourself: Delete terraform.tfstate from a scratch project with live resources, then run plan. Terraform will propose creating everything that already exists — because as far as it knows, it manages nothing. That is the clearest demonstration of what state is for.

Common mistake: Treating state as a cache that can be regenerated. It cannot. The write-once values are gone, the ownership record is gone, and reconstructing it means importing every resource by hand.