Testing, Validation & CI/CD

The three testing tiers and which tool serves each, static analysis that costs nothing, and the pipeline design whose single most important property is passing a reviewed plan file to apply.

advanced 21 min lesson hands-on task included

Testing infrastructure code catches misconfiguration before it reaches an environment, and — more importantly — makes refactoring safe. The pipeline is where those checks become mandatory rather than optional.


Topic 1: The Three Tiers

  1. Unit — individual modules behave as intended in isolation.
  2. Integration — components interact correctly when composed.
  3. Acceptance — the whole stack provisions and functions for real.

Each tier costs more and runs less often. Most value comes from the cheapest tier, run constantly.


Topic 2: Static Checks — Run These on Every Commit

terraform fmt -check      # canonical formatting; non-zero exit if a file would change
terraform validate        # syntax and internal consistency
tflint                    # lint for best practices and provider-specific errors
checkov -d infra/         # security misconfiguration scanner

terraform validate verifies attribute names and value types. It does not contact providers or read state, which is what distinguishes it from plan and what makes it safe to run as an editor save hook. It does require an initialised working directory with plugins and modules installed.

terraform init -backend=false    # init for validation without touching the backend
terraform validate

That -backend=false trick is what lets you validate a module in CI without backend credentials — useful for pull requests from forks and for validating reusable modules that have no backend of their own.


Topic 3: Automated Integration Testing

A Go-based harness provisions real infrastructure, asserts against it, and tears it down:

func TestBucketExists(t *testing.T) {
  t.Parallel()

  opts := &terraform.Options{
    TerraformDir: "../infra",
    Vars: map[string]interface{}{"environment": "test"},
  }
  defer terraform.Destroy(t, opts)

  terraform.InitAndApply(t, opts)

  bucketID := terraform.Output(t, opts, "bucket_id")
  assert.True(t, aws.DoesS3BucketExist(t, region, bucketID))
}

Note defer terraform.Destroy on the line before apply — it runs even if an assertion fails, which is what stops a failed test leaving resources billing indefinitely.

This is genuine end-to-end validation: it creates and destroys real resources, so it costs real money and real minutes. Run it on module releases and on a schedule, not on every push.


Topic 4: Pipeline Shape

name: Terraform

on:
  pull_request:
    paths: ['infra/**']
  push:
    branches: [main]

jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.9.5          # pin it

      - run: terraform init infra/
      - run: terraform fmt -check infra/
      - run: terraform validate infra/
      - run: terraform plan -input=false -no-color -out=plan.tfplan infra/

      - uses: actions/upload-artifact@v4
        with:
          name: tfplan
          path: infra/plan.tfplan

  apply:
    needs: plan
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production          # gates on manual approval
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - uses: actions/download-artifact@v4
        with: { name: tfplan }

      - run: terraform init infra/
      - run: terraform apply -input=false plan.tfplan

The same shape in a declarative pipeline DSL:

pipeline {
  agent any
  stages {
    stage('Init')     { steps { sh 'terraform init' } }
    stage('Validate') { steps { sh 'terraform validate' } }
    stage('Plan')     { steps { sh 'terraform plan -out=plan.tfplan' } }
    stage('Apply') {
      when { branch 'main' }
      steps { sh 'terraform apply plan.tfplan' }
    }
  }
}

Or a stage-based YAML pipeline:

image: hashicorp/terraform:latest
stages: [validate, plan, apply]

before_script: [terraform init]

validate:
  stage: validate
  script: [terraform validate]

plan:
  stage: plan
  script: [terraform plan -out=plan.tfplan]
  artifacts: { paths: [plan.tfplan] }
  only: [merge_requests]

apply:
  stage: apply
  script: [terraform apply plan.tfplan]
  only: [main]
  when: manual

Topic 5: The Property That Matters Most

Pass the reviewed plan file to apply. Everything else in the pipeline is convenience; this is the safety guarantee.

terraform apply plan.tfplan      # executes exactly this reviewed plan
terraform apply -auto-approve    # computes a fresh plan and runs it unseen

With a plan file, if reality moved between plan and apply — someone else applied, a resource was deleted — Terraform errors out rather than doing something nobody reviewed. With -auto-approve, it silently does whatever the new plan says.

The corollaries:

  • Separate plan and apply into distinct jobs, so a pull request can never apply.
  • Pin the Terraform version in the pipeline. Unpinned means behaviour changes under you on a day you changed nothing.
  • Use a dedicated CI service account with least-privilege credentials from the pipeline’s secret store.
  • Approval gates for production. An explicit human confirmation, via a protected environment or a manual job.
  • Separate pipelines or state per environment, so a staging run cannot reach production.

Topic 6: Reviewing a Plan in a Pull Request

The plan artifact is only useful if someone reads it. Two practices make that likely:

terraform show -no-color plan.tfplan > plan.txt   # post as a PR comment
terraform show -json plan.tfplan | jq '.resource_changes[]
  | select(.change.actions[] | contains("delete"))'

That second command extracts only the destructive changes. Posting that to a pull request focuses review on the part that can cause an outage, rather than on four hundred lines of unchanged attributes.


Try it yourself: Generate a plan file, apply something else out of band, then try to apply the stale plan. Read the error. Now do the same with -auto-approve and watch it proceed regardless. That contrast is the entire argument for plan artifacts.

Common mistake: A single pipeline job that runs terraform apply -auto-approve on merge. It looks like the same thing as plan-then-apply and is not: nobody ever reads a plan, and the apply executes whatever the world looks like at that moment rather than what was reviewed.