How to Control Kubernetes Costs: Resource Limits, Autoscaling, and Spot Nodes

Reduce Kubernetes infrastructure costs by 40-65% with proper resource requests, cluster autoscaling, spot node pools, and namespace quotas.

Kubernetes clusters are powerful but expensive. Controlling their costs requires focused attention on three levers: resource limits per pod, dynamic autoscaling of workloads, and strategic use of spot instances for node pools. This page guides engineers through configuring and maintaining a cost-efficient Kubernetes environment using only real, standard tools: kubectl, curl, docker, and systemctl, with no vendor-specific magic.

Right-Sizing Pods with Resource Limits

Resource limits define how much CPU and memory a container can consume. Misconfigured limits lead to wasted resources or performance degradation. The key is to set both requests and limits in the pod spec, ensuring the scheduler places pods optimally and the kernel enforces bounds.

A pod with mismatched requests and limits may be scheduled on a node with insufficient capacity, yet still exceed its memory budget.

Setting CPU and Memory Limits

In a pod specification, resources block defines how much of each resource a container needs. The requests field affects scheduling, while limits enforce runtime constraints.

apiVersion: v1
kind: Pod
metadata:
  name: web-server
spec:
  containers:
    - name: app
      image: nginx:1.21
      resources:
        requests:
          memory: "128Mi"
          cpu: "100m"
        limits:
          memory: "512Mi"
          cpu: "500m"

Monitoring Over-Commitment

Use kubectl top pods to observe actual CPU and memory usage:

kubectl top pods --all-namespaces

If a pod consistently uses 450m CPU but has a limit of 500m, it’s underutilized. If it spikes to 550m, it’s throttled. Use kubectl describe pod <name> to see if the pod is being throttled:

Containers:
  app:
    State:          Running
    CPU Requests:   100m, CPU Limits: 500m
    Memory Requests: 128Mi, Memory Limits: 512Mi
    Allocated resources:
      CPU:     475m
      Memory:  490Mi
    Events:
      Type     Reason     Age                From               Message
      ----     ------     ----               ----               -------
      Warning  OOMKilled  1m                 kubelet            Container app failed: Memory limit exceeded

When OOMKilled appears, the container exceeded its memory limit. Use kubectl logs <pod-name> --container=app to check for OOM-related errors in application logs.

Tuning Limits for Cost Efficiency

Set requests to match typical usage and limits to allow headroom during traffic bursts. For a web service, set:

Use kubectl exec to run diagnostic commands:

kubectl exec -it web-server-12345 -- sh -c "ps aux --sort=-%mem | head -10"

Then adjust the limits.memory in the deployment manifest and apply:

kubectl apply -f deployment.yaml

Common Pitfalls

Autoscaling Workloads with Horizontal Pod Autoscaler

Horizontal Pod Autoscaler (HPA) scales the number of pod replicas based on metrics. It reads metrics from the metrics server and adjusts the replica count.

Configuring HPA with CPU and Custom Metrics

Define HPA using kubectl create hpa or a YAML manifest.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-server-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-server
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

This HPA scales the web-server deployment to maintain 70% average CPU utilization and ensures 100 requests per second across all pods.

Verifying HPA Behavior

Use kubectl get hpa to check current status:

kubectl get hpa web-server-hpa -o wide

Output:

NAME                REFERENCE           TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
web-server-hpa      Deployment/web-server   68%/70%, 85/100   3         20        6          2m

Use kubectl describe hpa to inspect scaling decisions:

kubectl describe hpa web-server-hpa

Look for events like:

Events:
  Type     Reason            Age                From                       Message
  ----     ------            ----               ----                       -------
  Normal   SuccessfulRescale 1m                 horizontal-pod-autoscaler  New size: 8; reason: cpu resource utilization (current 72% > desired 70%)
  Normal   ScalingReplicaSet 30s                horizontal-pod-autoscaler  Scaled up replica set web-server-7890 to 8

Tuning HPA for Stability

behavior:
  scaleUp:
    stabilizationWindowSeconds: 60
    policies:
      - type: Pods
        value: 5
        periodSeconds: 30
  scaleDown:
    stabilizationWindowSeconds: 300
    policies:
      - type: Percent
        value: 10
        periodSeconds: 60

Failure Modes

Best Practices

Deploying Spot Nodes for Cost Efficiency

Spot nodes are pre-emptible instances that cost less than on-demand instances. Use them for stateless workloads with tolerance for interruptions.

Creating a Spot Node Pool

Use kubectl to label nodes and create a nodeSelector in pod specs.

apiVersion: v1
kind: Pod
metadata:
  name: worker-job
spec:
  nodeSelector:
    kubernetes.io/instance-type: m5.large
    beta.kubernetes.io/lifecycle: spot
  containers:
    - name: processor
      image: processor:latest

The nodeSelector ensures pods run on nodes labeled with beta.kubernetes.io/lifecycle: spot.

Labeling Nodes for Spot Instances

Label existing nodes:

kubectl label nodes ip-10-0-1-10 beta.kubernetes.io/lifecycle=spot
kubectl label nodes ip-10-0-1-11 beta.kubernetes.io/lifecycle=spot

Use kubectl get nodes -o wide to verify:

NAME           STATUS   ROLES    AGE   VERSION   LABELS
ip-10-0-1-10   Ready    <none>   2d    v1.25.0   beta.kubernetes.io/lifecycle=spot,kubernetes.io/instance-type=m5.large
ip-10-0-1-11   Ready    <none>   2d    v1.25.0   beta.kubernetes.io/lifecycle=spot,kubernetes.io/instance-type=m5.large

Handling Preemption

Spot nodes can be terminated with a 2-minute warning. The pod must be resilient to SIGTERM and SIGKILL.

Use lifecycle in pod spec:

lifecycle:
  preStop:
    exec:
      command:
        - sh
        - -c
        - |
          echo "Starting graceful shutdown..."
          sleep 30
          echo "Shutdown complete."

This ensures the pod has 30 seconds to finish work before termination.

Monitoring Node Termination

Use kubectl get events to track preemption:

kubectl get events --field-selector=type=Normal,reason=NodeNotReady --watch

Look for:

Type      Reason           Age                From               Message
----      ------           ----               ----               -------
Normal    NodeNotReady     1m                 node-controller    Node ip-10-0-1-10 status is now: NotReady
Normal    NodeReady        1m                 node-controller    Node ip-10-0-1-10 status is now: Ready
Normal    NodePreempted    30s                node-controller    Node ip-10-0-1-10 preempted by spot instance

Failure Modes

Best Practices

Putting It All Together

A cost-efficient Kubernetes cluster integrates all three: resource limits, autoscaling, and spot nodes.

  1. Define resource requests and limits in all pod specs.
  2. Enable HPA on critical workloads with CPU and custom metrics.
  3. Label spot nodes and use nodeSelector to target them.
  4. Implement graceful shutdown in preStop hooks.
  5. Monitor kubectl describe hpa, kubectl top pods, and kubectl get events.

This combination ensures that every dollar spent on compute is well utilized, with minimal waste and predictable performance.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.