Autoscaling: HPA, VPA, Cluster Autoscaler & Karpenter

Three autoscalers exist and they scale different things: HPA adds pods, VPA resizes pods, Cluster Autoscaler / Karpenter adds nodes. Running the wrong combination produces oscillation; running none produces either waste or an outage.

Topic 1: HPA — More Pods

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 50
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100          # may double
          periodSeconds: 30
        - type: Pods
          value: 4            # ...or add 4, whichever is larger
          periodSeconds: 30
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300      # look back 5 min before shrinking
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

The formula:

desiredReplicas = ceil( currentReplicas × ( currentMetric / targetMetric ) )

10 pods averaging 90% against a 70% target → ceil(10 × 90/70) = 13 pods.

Why your HPA shows <unknown> — two causes, in order of likelihood:

1. No resource requests. averageUtilization is a percentage of the request. With no request there is no denominator, and the HPA cannot compute anything.

kubectl get hpa api
# TARGETS: <unknown>/70%
kubectl get deploy api -o jsonpath='{.spec.template.spec.containers[*].resources.requests}{"\n"}'
# {}       ← there it is

2. metrics-server is not installed or not working.

kubectl top pods            # if this fails, the HPA cannot work either
kubectl get apiservices | grep metrics
kubectl -n kube-system logs deploy/metrics-server

metrics-server is not installed by default on many distributions, and it is a hard dependency for CPU/memory HPAs.

If any container in the pod lacks a request for the metric in question, the whole pod is excluded from the calculation — a sidecar without requests can therefore break an HPA that looks correctly configured.

Scaling on something other than CPU:

CPU is a poor proxy for load on an I/O-bound service. Custom and external metrics are usually better:

  metrics:
    - type: Pods                        # per-pod, from your app
      pods:
        metric: { name: http_requests_per_second }
        target: { type: AverageValue, averageValue: "500" }
    - type: External                    # from outside the cluster
      external:
        metric:
          name: sqs_queue_depth
          selector: { matchLabels: { queue: orders } }
        target: { type: AverageValue, averageValue: "30" }

These require an adapter implementing the custom/external metrics API — Prometheus Adapter or KEDA. KEDA is worth knowing about specifically: it provides dozens of ready-made scalers (Kafka lag, SQS depth, Redis list length, cron) and can scale to zero, which plain HPA cannot (minReplicas must be ≥1 without the alpha gate).

The behavior block prevents thrashing:

Scale up fast, scale down slowly — that asymmetry is deliberate. stabilizationWindowSeconds: 300 on scale-down makes the HPA use the highest recommendation from the last five minutes, so a brief dip in traffic does not immediately remove capacity you will need again in ninety seconds.

Topic 2: VPA — Right-Sized Pods

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  updatePolicy:
    updateMode: "Off"        # Off | Initial | Recreate | Auto
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed: { cpu: 50m,  memory: 64Mi }
        maxAllowed: { cpu: "4",  memory: 8Gi }
        controlledResources: ["cpu", "memory"]

kubectl describe vpa api | grep -A12 'Recommendation'

Use updateMode: "Off" as a recommender. It analyses real usage and tells you what to set, without the disruption of evicting pods. That is the highest-value, lowest-risk use of VPA, and it solves the “what numbers should I put here” problem from the resources lesson.

Do not run VPA in Auto mode on the same workload as an HPA on CPU or memory. They fight: VPA raises requests → utilisation percentage drops → HPA scales in → load per pod rises → VPA raises requests again. Either use VPA on memory while the HPA uses CPU, or keep VPA in recommend-only mode.

In-place pod resizing (beta in 1.33) is beginning to remove the eviction requirement, which will make VPA considerably more usable. Until it is stable, Recreate means real disruption.

Topic 3: Cluster Autoscaler and Karpenter — More Nodes

Neither HPA nor VPA creates capacity. When pods cannot schedule, something must add nodes.

Cluster Autoscaler

Watches for Pending pods and scales a node group (ASG / MIG / VMSS).

• Scale up: a pod is unschedulable → find a node group whose template would fit it → increase its size.
• Scale down: a node is under-utilised (default below 50%) for 10 minutes and its pods can move → drain and remove it.

It will not remove a node if any pod there:

• Has no controller (a bare pod)
• Uses local storage (emptyDir) — unless annotated safe-to-evict
• Is blocked by a PodDisruptionBudget
• Is in kube-system without a PDB
• Has the annotation cluster-autoscaler.kubernetes.io/safe-to-evict: "false"

A single such pod pins an entire node indefinitely, which is a common and expensive surprise.

kubectl -n kube-system logs deploy/cluster-autoscaler | grep -E 'scale_up|scale_down|unremovable'
kubectl get configmap cluster-autoscaler-status -n kube-system -o yaml

That ConfigMap is the fastest way to see what the autoscaler thinks and why it declined to act.

Karpenter

A different model: rather than resizing pre-defined node groups, it looks at the actual requirements of pending pods and provisions a right-sized instance directly.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
      nodeClassRef:
        name: default
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s
    expireAfter: 720h        # recycle nodes every 30 days for patching
  limits:
    cpu: "1000"

consolidationPolicy: WhenEmptyOrUnderutilized is Karpenter’s real advantage — it continuously repacks workloads onto fewer, better-fitting nodes. It is also why PDBs matter enormously with Karpenter: consolidation moves pods constantly, and a workload with no PDB can be disrupted at any time.

expireAfter is worth setting regardless: forcing node replacement on a schedule means your AMI patching happens continuously rather than as a scary quarterly event.

Topic 4: How They Fit Together

Traffic rises
   → HPA adds pods
      → pods are Pending (no capacity)
         → Cluster Autoscaler / Karpenter adds nodes
            → pods schedule
Traffic falls
   → HPA removes pods (after stabilisation)
      → nodes become under-utilised
         → autoscaler drains and removes nodes (respecting PDBs)

Requirements for that chain to work:

1. Resource requests on everything — the HPA needs them for percentages, the node autoscaler needs them for sizing.
2. A PDB on every meaningful workload — otherwise node scale-down disrupts you freely.
3. maxReplicas you have actually thought about — an HPA with maxReplicas: 1000 and a runaway metric will happily provision a very expensive cluster.
4. Sane minReplicas — 1 means no redundancy during a node drain.

Topic 5: Debugging

kubectl get hpa api
# NAME  REFERENCE         TARGETS          MINPODS  MAXPODS  REPLICAS
# api   Deployment/api    45%/70%, 60%/80%  3        50       5

kubectl describe hpa api        # ScalingActive / AbleToScale / ScalingLimited conditions
kubectl get events --field-selector involvedObject.kind=HorizontalPodAutoscaler

# Is the metrics pipeline alive at all?
kubectl top pods
kubectl get --raw /apis/metrics.k8s.io/v1beta1/pods | jq '.items | length'
kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq '.resources[].name'

Topic 6: Advice

1. Set requests before anything else. Every autoscaler depends on them.
2. Use VPA in recommend-only mode to choose those requests from real data.
3. Scale on a metric that reflects user experience — queue depth or RPS beats CPU for most services.
4. Scale up fast, down slow. The cost of over-provisioning for five minutes is far lower than the cost of a cold start under load.
5. minReplicas ≥ 2 for anything that must survive a node drain, and ≥3 if you have a PDB requiring 2 available.
6. Pre-scale for known events. Autoscaling reacts; a marketing campaign at 09:00 wants a scheduled scale-up at 08:45.
7. Consider KEDA if your load is event-driven — scaling on queue depth with scale-to-zero is a materially different cost profile.

Try it yourself: Create an HPA on a Deployment with no resource requests. Confirm TARGETS: <unknown>. Add requests and watch the same HPA start working with no other change. That single dependency explains most non-functioning HPAs.

Common mistake: Setting averageUtilization: 90 to “use resources efficiently”. The HPA only acts after the threshold is crossed, and new pods take time to schedule, pull and become ready. At 90% you begin scaling when you are already saturated, and users see the latency for the entire startup window. 60–70% leaves room for the response to land before it matters.