Supply Chain Security in the Pipeline

Most pipelines scan and report. A supply chain is only defended at the point where something is refused — and in a Kubernetes estate, that point is admission control, not the build.

Topic 1: The Chain, and Where It Is Enforced

Four links produce claims; admission control is the only place any of them is enforced. The banner at the bottom is the sentence to take away.

Each link in the chain produces a claim somebody can check later:

Source — signed commits, protected branches, reviewed changes. The Git module covers this; the relevant point here is that a pipeline can only be as trustworthy as the commit it started from.

Build — an isolated environment, pinned tools, no ambient credentials. Ideally hermetic: no network access during the build, so a dependency cannot be swapped mid-build.

SBOM — a machine-readable list of what is actually inside the artifact.

Signature and attestation — a cryptographic statement that this build produced this digest.

And then enforcement, which is the one that changes outcomes:

scan and warn        → a report nobody reads
scan and fail        → the build stops (good)
sign and verify      → the cluster refuses unsigned images (better)
both                 → what you want

Topic 2: SBOM and Vulnerability Gating

stage('SBOM and scan') {
    steps {
        sh '''
          set -euo pipefail
          syft "${IMAGE}@${DIGEST}" -o spdx-json > sbom.json
          grype sbom:sbom.json --fail-on high --only-fixed -o table
        '''
        archiveArtifacts artifacts: 'sbom.json'
    }
}

Two flags carry most of the judgement:

--fail-on high — the threshold at which the build stops. Set it, argue about it once, and write it down. A pipeline that fails on medium in an ecosystem with hundreds of medium advisories will be bypassed within a fortnight.

--only-fixed — fail only on vulnerabilities that have a fix available. Failing a build for an unfixable CVE gives the team no action except to disable the check, which is the worst outcome.

Keep the SBOM as an artifact, and attach it to the image, so that when a new CVE is announced you can answer “are we affected” by querying stored SBOMs rather than rebuilding and rescanning everything:

cosign attach sbom --sbom sbom.json "${IMAGE}@${DIGEST}"
cosign attest --predicate sbom.json --type spdxjson "${IMAGE}@${DIGEST}"

An exception process is required, not optional: a documented allow-list with an owner and an expiry date, in the repository, reviewed. Without one, the first urgent release with an unfixable finding removes the gate permanently.

Topic 3: Signing and Provenance

Keyless signing ties the signature to a workload identity rather than to a key somebody has to store:

stage('Sign') {
    steps {
        sh '''
          set -euo pipefail
          # Jenkins with an OIDC-federated identity, or Cloud Build's service account
          cosign sign --yes "${IMAGE}@${DIGEST}"
          cosign verify "${IMAGE}@${DIGEST}" \
            --certificate-identity-regexp='.*' \
            --certificate-oidc-issuer=https://accounts.google.com
        '''
    }
}

There is no private key to protect, rotate or leak. The signature says “this identity, in this pipeline, signed this digest”, and the verification policy names which identity you accept.

Provenance goes further: a signed statement of how the artifact was built — the source commit, the builder, the parameters. Cloud Build produces SLSA provenance automatically for images declared in images:; for Jenkins you generate it explicitly with cosign attest or a SLSA generator.

# What was this built from? Answerable months later.
cosign verify-attestation "${IMAGE}@${DIGEST}" --type slsaprovenance \
  --certificate-oidc-issuer=https://accounts.google.com \
  --certificate-identity-regexp='.*' | jq '.payload | @base64d | fromjson'

The SLSA levels are a useful ladder rather than a compliance exercise: scripted build (L1), signed provenance from a hosted builder (L2), hardened and non-falsifiable builder (L3). Most teams should aim at L2 — it is achievable with Cloud Build or GitHub Actions and a cosign attest step, and it makes the provenance question answerable.

Topic 4: Deploy by Digest

image: ghcr.io/acme/checkout:1.6.0                      ← a tag can be moved
image: ghcr.io/acme/checkout@sha256:9f8e7d…             ← this cannot

A tag is a mutable pointer. Between your scan and your deployment, :1.6.0 can be repushed — by an attacker with registry credentials, or by a colleague fixing something. Everything the pipeline proved was about a digest; deploy that digest.

stage('Build and capture the digest') {
    steps {
        script {
            env.DIGEST = sh(returnStdout: true, script: '''
                crane digest "${IMAGE}:${GIT_COMMIT}"
            ''').trim()
        }
        echo "Deploying ${IMAGE}@${env.DIGEST}"
    }
}

Pass the digest to whatever deploys — Helm --set image.digest, Cloud Deploy --images, a Kustomize image transform. Every one of them supports it, and using a tag instead is the most common gap between a “secured” pipeline and an actually secured one.

Topic 5: Admission Control — the Enforcement Point

Everything above is a claim. The cluster refusing to run an artifact without a valid claim is the control.

Kyverno:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: { name: require-signed-images }
spec:
  validationFailureAction: Enforce
  rules:
    - name: verify-signature
      match:
        any:
          - resources: { kinds: [Pod], namespaces: [prod] }
      verifyImages:
        - imageReferences: ['ghcr.io/acme/*']
          attestors:
            - entries:
                - keyless:
                    issuer: https://accounts.google.com
                    subject: 'clouddeploy@acme.iam.gserviceaccount.com'

Binary Authorization is the managed GCP equivalent, with attestations required per environment and Cloud Build as an attestor.

Two operational realities of rolling this out:

Start in audit mode. validationFailureAction: Audit reports what would be rejected. Run it for a fortnight, fix what appears, then switch to Enforce. Turning it on cold will block something at the worst moment.
Exempt what must be exempt, explicitly. kube-system, the CNI, the monitoring agent — third-party images that will never carry your signature. An exemption list in the policy is honest; a policy quietly disabled because it broke the cluster is not.

Topic 6: What This Looks Like as a Pipeline

stages {
    stage('Build')  { steps { sh 'make build' } }                       // pinned tools
    stage('Test')   { steps { sh 'make test' } }
    stage('Image')  { steps { sh 'make image' } }                       // kaniko, no docker socket
    stage('SBOM')   { steps { sh 'syft $IMAGE@$DIGEST -o spdx-json > sbom.json' } }
    stage('Scan')   { steps { sh 'grype sbom:sbom.json --fail-on high --only-fixed' } }
    stage('Sign')   { steps { sh 'cosign sign --yes $IMAGE@$DIGEST' } }
    stage('Attest') { steps { sh 'cosign attest --predicate sbom.json --type spdxjson $IMAGE@$DIGEST' } }
    stage('Deploy') { steps { sh 'gcloud deploy releases create … --images=app=$IMAGE@$DIGEST' } }
}

Seven stages, six of which are one command. The expensive parts are not the tools:

Agreeing the CVE threshold and the exception process.
Getting the identity right so signing is keyless rather than another stored secret.
The audit-then-enforce rollout of admission control.
Removing the tag-based deploys that already exist.

Try it yourself: plant a known-vulnerable dependency, confirm the build fails, then add it to an allow-list with an expiry and confirm it passes. Doing both halves is what makes the gate sustainable — a gate with no exception path gets removed.

Common mistake: adding a scanning stage that reports findings and always exits zero, then reporting that the pipeline has security scanning. Nothing has changed: no build is blocked, nobody reads the output after week two, and the first real finding ships. Either the build fails on the finding or admission rejects the image — anything else is a dashboard.