Kubernetes Node Optimization: Karpenter & Autoscaling

Scaling node groups in Kubernetes using traditional AWS Auto Scaling Groups (ASGs) and the cluster-autoscaler is slow and inefficient. Node group adjustments are restricted to rigid instance scaling configurations, leading to poor bin-packing and high waste.

This guide outlines how to implement high-speed node provisioning and cost optimization utilizing Karpenter, the open-source Kubernetes autoscaler.

---

1. Why Karpenter Over Cluster-Autoscaler

Traditional cluster-autoscaler hooks directly into cloud provider scaling API layers (such as AWS ASGs). Under resource constraints:

1. A pod goes into Pending state.
2. Cluster-autoscaler polls the cluster state and requests the ASG to increment its desired instance count.
3. The ASG spins up an identical instance. This process takes 3-5 minutes and offers no instance flexibility.

In contrast, Karpenter acts directly on the Kubernetes scheduler level, bypasses ASGs, and calls AWS EC2 fleet APIs directly:

Unscheduled Pod (Stuck Pending)
  -> Karpenter scans Pod's requests, limits, & node selectors
  -> Evaluates ideal EC2 instance type (bin-packing computation)
  -> Directly launches the target EC2 node (30-40 seconds)

This model permits instant consolidation and dynamic instance-type swapping.

---

2. Configuring Karpenter NodePools

To configure Karpenter, define a NodePool resource. This config specifies the provisioning constraints and constraints for scheduling nodes:

# nodepool.yaml
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: karpenter.category
          operator: In
          values: ["c", "m", "r"] # Compute, Memory, General Purpose
      nodeClassRef:
        name: default-ec2nodeclass
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h # 30 Days

Key Parameters Explained:

• Consolidation Policy (WhenUnderutilized): Karpenter automatically tracks node utilization and consolidates workloads onto smaller or fewer instances when resource consumption drops.
• Capacity Types: Specifying both spot and on-demand enables automated spot fallback. Karpenter prioritizes spot nodes for batch processing or non-critical environments while maintaining stability on-demand for production endpoints.

---

3. Bin-Packing Strategies & Workload Placement

Efficient resource consumption requires strict pod resource declarations. Without resource requests, the scheduler cannot calculate optimal node layouts.

3.1 Workload QoS Alignment

Organize nodes based on Quality of Service (QoS) classes:

# Pod Spec QoS Definition Example
apiVersion: apps/v1
kind: Deployment
metadata:
  name: billing-service
spec:
  template:
    spec:
      containers:
      - name: app
        image: billing-gateway:v1.2
        resources:
          limits:
            cpu: "2"
            memory: "4Gi"
          requests:
            cpu: "1"
            memory: "2Gi"

If requests and limits match exactly, the pod runs as Guaranteed, preventing eviction unless the host node suffers a physical failure.

3.2 Taints and Tolerations for Spot Instances

To isolate critical workloads from spot disruptions, taint spot node instances and require pods to declare corresponding tolerations:

# Deployment Toleration config
tolerations:
- key: "karpenter.sh/capacity-type"
  operator: "Equal"
  value: "spot"
  effect: "NoSchedule"

This configuration keeps core API controllers on stable on-demand nodes, while ephemeral analysis runners are scheduled on cheap spot instances.