Init Containers & Native Sidecars

Init containers have been around since the beginning. Native sidecars are new — stable in Kubernetes 1.33 — and they fix a class of problem that the ecosystem had been working around with increasingly elaborate hacks for years.

Topic 1: Init Containers

Init containers run to completion, in order, before any app container starts. If one fails, the pod restarts it (subject to restartPolicy) and no app container runs.

apiVersion: v1
kind: Pod
spec:
  initContainers:
    - name: wait-for-db
      image: busybox:1.36
      command:
        - sh
        - -c
        - |
          until nc -z postgres 5432; do
            echo "waiting for postgres..."; sleep 2
          done
    - name: run-migrations
      image: myapp:1.4.2
      command: ["/app/migrate", "up"]
      envFrom:
        - secretRef: { name: db-credentials }
  containers:
    - name: api
      image: myapp:1.4.2

kubectl get pod shows progress as Init:0/2 → Init:1/2 → PodInitializing → Running.

What they are genuinely good for:

• Waiting for a dependency to be reachable before starting.
• Schema migrations — though see the warning below.
• Fetching config or secrets into a shared emptyDir at startup.
• Setting permissions on a mounted volume that the app cannot chown itself.
• Doing privileged setup so the app container never needs the privilege:

  initContainers:
    - name: sysctl
      image: busybox:1.36
      securityContext:
        privileged: true          # ONLY the init container is privileged
      command: ["sh","-c","sysctl -w vm.max_map_count=262144"]
  containers:
    - name: elasticsearch
      securityContext:
        privileged: false         # the long-running container is not

That split is a real security win: elevated permission exists for two seconds instead of for the pod’s lifetime.

Where they go wrong:

Migrations in an init container run once per pod. A 5-replica Deployment runs the migration 5 times, concurrently, on every rollout. If your migration tool has proper locking that is merely wasteful; if it does not, it is a corrupted schema. Migrations belong in a Job that runs once, gated before the rollout.

A failing init container blocks forever, silently. Init:0/1 for an hour looks like a scheduling problem to the untrained eye. Read its logs specifically:

kubectl logs api-xyz -c wait-for-db
kubectl logs api-xyz -c wait-for-db --previous

Resource accounting is different. The pod’s effective request is max(largest init container, sum of app containers), because init containers run sequentially and not concurrently with the app. A heavyweight init container can therefore raise the whole pod’s scheduling footprint.

Topic 2: The Sidecar Problem That Existed Until 1.33

A “sidecar” was never a Kubernetes concept — it was just a second container in containers:. That informal status caused four concrete, well-known failures:

1. Startup race. All containers in containers: start in parallel. An app that immediately makes a network call could start before the Istio/Linkerd proxy was ready, and its first requests failed. The ecosystem worked around this with holdApplicationUntilProxyStarts, custom entrypoint wrappers, and retry loops.

2. Jobs never completed. A Job’s pod finishes when all containers exit. The app container exits 0; the proxy runs forever; the pod stays Running and the Job never completes. Workarounds included the app curl-ing the proxy’s /quitquitquit endpoint on exit — genuinely what people shipped.

3. Shutdown ordering. On termination, all containers get SIGTERM together. The proxy could die first, so the app’s in-flight requests and its final telemetry flush failed.

4. No ordering guarantees at all, so log shippers could miss the app’s startup logs.

Topic 3: Native Sidecars — the Fix

A native sidecar is an init container with restartPolicy: Always. That one field changes its semantics entirely.

apiVersion: v1
kind: Pod
spec:
  initContainers:
    - name: proxy
      image: envoyproxy/envoy:v1.31
      restartPolicy: Always          # ← THIS makes it a sidecar
      startupProbe:
        httpGet: { path: /ready, port: 15021 }
        periodSeconds: 1
        failureThreshold: 30
    - name: log-shipper
      image: fluent-bit:3.1
      restartPolicy: Always
      volumeMounts:
        - name: logs
          mountPath: /logs
  containers:
    - name: api
      image: myapp:1.4.2
      volumeMounts:
        - name: logs
          mountPath: /var/log/app
  volumes:
    - name: logs
      emptyDir: {}

Semantics, and how each fixes a numbered problem above:

The startup gate is startupProbe on the sidecar. The next container does not start until the sidecar’s startup probe passes — which is exactly the ordering guarantee meshes previously had to fake.

Version requirement: stable in 1.33, beta and on by default from 1.29. On 1.28 or older the field is ignored and you get a plain init container that blocks forever. Check before adopting:

kubectl version --short

Topic 4: Sidecar Patterns

Service mesh proxy — intercepts all traffic for mTLS, retries, telemetry. The canonical sidecar, and the one native sidecars were largely designed for.

Log shipper — tails a shared emptyDir. Worth noting this is often unnecessary: if your app logs to stdout, the container runtime already captures it and a node-level DaemonSet collector picks it up with one agent per node instead of one per pod. A per-pod log sidecar on 500 pods is 500 extra containers.

Config reloader — watches a ConfigMap mount and signals the app (nginx -s reload) when it changes, solving the “mounted config updated but the app did not notice” problem from the config lesson.

Credential refresher — fetches short-lived credentials from Vault or a cloud metadata service and writes them to a shared volume on a timer.

Ambassador — the app connects to localhost:6379 and the sidecar handles sharding, TLS or failover.

Topic 5: Choosing Between Them

The last row is worth repeating from the pods lesson: if the two things do not need to share a network namespace and a machine, they are two workloads.

Topic 6: Debugging Multi-Container Pods

Every command that touches a container needs -c once there is more than one:

kubectl logs api-xyz -c proxy
kubectl logs api-xyz -c proxy --previous
kubectl logs api-xyz --all-containers=true --prefix=true    # everything, labelled
kubectl exec -it api-xyz -c proxy -- sh
kubectl describe pod api-xyz            # per-container state and reasons

kubectl get pod api-xyz -o jsonpath='{range .status.initContainerStatuses[*]}{.name}{"\t"}{.state}{"\n"}{end}'
kubectl get pod api-xyz -o jsonpath='{range .status.containerStatuses[*]}{.name}{"\t"}{.ready}{"\n"}{end}'

--all-containers --prefix is the one to remember when you do not yet know which container is the problem — it interleaves every container’s output with a label on each line.

Reading pod status with sidecars:

Native sidecars appear under initContainerStatuses, not containerStatuses, which surprises people writing automation. The READY column in kubectl get pods counts only app containers — so a pod showing 1/1 may have a crash-looping sidecar you would only see in describe.

kubectl get pod api-xyz -o jsonpath='{.status.initContainerStatuses[*].restartCount}{"\n"}'

Try it yourself: Create a Job whose pod has a regular sidecar container running sleep infinity. Confirm the Job never completes. Move that container to initContainers with restartPolicy: Always and confirm the Job completes normally.

Common mistake: Adding restartPolicy: Always to an init container on a cluster older than 1.29. The field is silently ignored, so you get a plain init container that never exits — and the pod stays in Init:0/1 forever with no error explaining why. Check the cluster version before using the pattern.