Container Orchestration Patterns

Container Orchestration Patterns

Container orchestration patterns capture the recurring ways platform teams arrange workloads, updates, and self-healing logic on top of a cluster manager. They matter whenever you’re running more than a handful of containers in production and need guarantees about availability, update safety, and resource fairness. This page collects the patterns you’ll actually invoke, the commands you’ll type, and the failure modes that catch teams off guard.

Rolling Updates and Controlled Replacement

The most common day-to-day pattern is replacing pods one at a time so that the service remains available throughout. In a kubectl-driven workflow the canonical command sequence looks like this:

kubectl set image deployment/app container=registry/example/app:v2 --record

# Watch the rollout progress, failing after a reasonable deadline
kubectl rollout status deployment/app --timeout=300s

If the rollout stalls, the annotated revision lets you reverse course immediately:

kubectl rollout undo deployment/app

Exact error when a health check blocks progress

error: deployment "app" has timed out waiting for rollout to finish:
5 of 5 old replicas are available, 0 new replicas are available:
Ingress failed health check

Sharp edges

• Image pull backoff after a tag push can make it appear that the rollout is stuck when the real issue is registry authentication or rate limiting.
• Version skew between the new pod’s binary and the old one’s database schema can cause silent failures that pass liveness probes but break request logic.
• Probe misconfiguration is the top culprit: a liveness probe that is too aggressive (e.g., initialDelaySeconds: 0 or periodSeconds: 1) will kill pods during startup, causing a rapid oscillation between old and new replicas and ultimately a rollout failure with the error above. Conversely, a readiness probe that never passes will prevent traffic from ever reaching the new version, making it look like the deployment hung.

Canary and Blue-Green Deployments

When you need to validate a new version with a subset of traffic before full cutover, the pattern shifts from automatic replacement to deliberate traffic routing. The mechanics are the same across orchestrators: adjust the set of endpoints a Service selects, or use an ingress/controller weight.

Concrete command (adjusting Service selectors)

# Route 10% of traffic to the canary version by mutating the Service’s selector
kubectl patch service app -p '{"spec":{"selector":{"version":"canary","track":"stable"}}}'

In practice, most teams rely on a service mesh (e.g., Istio, Linkerd) or cloud provider ingress (AWS ALG, GCP Ingress) to split traffic via request.weight or percent annotations. The underlying idea is identical: the orchestrator continues to manage pod lifecycles, while an external routing layer directs requests.

Sharp edges

• Sticky sessions tied to a user’s IP or cookie can defeat a canary split, sending all of a user’s requests to the same version regardless of the weight.
• DNS TTL and client-side caching mean that traffic shifts may not be visible for minutes after the Service selector changes.
• Unexpected load spikes on the new version can occur if the canary inherits the same resource quotas as the stable set but a different autoscaling threshold, leading to OOMKills or throttling that are hard to debug without per-pod metrics.

Self-Healing: Liveness and Readiness Probes

Orchestrators distinguish between “the process died” and “the process is up but not yet serving traffic” through two complementary probe types. The YAML fragment below is representative of what you’ll place inside a pod spec:

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5

Use kubectl to inspect the current state of a pod’s probes:

kubectl describe pod <pod-name> | grep -A5 "Probes:"

Exact error when a liveness probe fails repeatedly

Back-off restarting failed container(example-app) 1m12s repeated message "Get "http://10.244.0.5:8080/healthz": dial tcp 10.244.0.5:8080: connect: connection refused"

Sharp edges

• Probes that depend on downstream services (databases, message queues) can cause a pod to be marked unready or unhealthy during infrastructure startup, even though the application code is functionally correct. This triggers unnecessary restarts and can cascade into cluster-wide restart storms.
• Readiness vs. liveness confusion is a frequent root cause of “my pod is running but I’m getting 502s.” If a readiness probe is missing or misconfigured, the Service will send traffic to a pod that hasn’t finished initializing, and the application will return errors because its internal state isn’t ready.
• Startup probe gap (absent in older orchestrator versions) leads teams to set very short initialDelaySeconds, which means the first probe fires before the app has loaded its configuration, resulting in immediate failure.

Resource Scheduling and Fairness

Preventing a single workload from starving others is achieved by expressing CPU and memory requests and limits. The orchestrator uses these values to place pods, decide QoS (Quality of Service) class, and make eviction decisions under pressure.

Representative pod spec fragment

resources:
  requests:
    cpu: "200m"
    memory: "128Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

Adjust these values on a running deployment:

kubectl edit deployment/app
# Modify the resources section, then save and exit

Sharp edges

• Requests over-committed across all pods on a node will cause the scheduler to reject new placements, but the error is often opaque: 0/3 nodes are available: 3 Insufficient cpu. Teams frequently mistake this for a cluster capacity issue when the real cause is a few pods with high requests and no limits.
• Limits without requests default the pod to “burstable” QoS, meaning the orchestrator will only kill it after other burstable pods have been evicted. If you expect OOMKill under pressure, you must set both.
• OOMKilled appears in kubectl get pods output as OOMKilled in the STATUS column, accompanied by Container terminated with exit code 137. This silently kills the process without a graceful shutdown hook, potentially corrupting in-memory state or leaving database connections in an inconsistent state.

Cluster Auto-Repair and Node Disruption

When a node hardware failure or scheduled maintenance occurs, the orchestrator must drain the node and reschedule its pods elsewhere. The standard toolchain involves cordoning, draining, and relying on the cluster autoscaler or manual intervention.

Concrete commands

# Prevent new pods from scheduling on the node
kubectl cordon node1

# Drain the node, respecting DaemonSets and deleting empty-dir volumes
kubectl drain node1 --ignore-daemonsets --delete-emptydir-data

If the cluster autoscaler is active, it will add a new node when the scheduler cannot find a fit:

# Watch the autoscaler log (cloud-provider-specific)
# Typical flag set at deployment:
# --scale-down-utilization-threshold=0.5

Sharp edges

• Pods with restartPolicy: Never (e.g., CI job pods) are not rescheduled after a node drain; they are simply lost. Teams often forget this and wonder why a job re-runs but never completes.
• Taints and tolerations can block drainage entirely. A pod with a taint that matches the node’s new taint (added during maintenance) will stay bound to the node, preventing the drain from completing and leaving the orchestrator in a retry loop.
• DaemonSets (logging, monitoring, node-exporter) are excluded by --ignore-daemonsets by default, but if a DaemonSet pod fails to restart on the new node, the cluster’s observability pipeline may break silently.

Configuration and Secret Injection

Getting configuration and secrets into pods without baking them into images is a first-class pattern. The orchestrator provides two primary mechanisms: ConfigMaps for plaintext key-value data and Secrets for base64-encoded sensitive data.

ConfigMap volume mount (representative YAML)

volumeMounts:
  - name: app-config
    mountPath: /etc/app-config
readOnly: true
volumes:
  - name: app-config
    configMap:
      name: app-config-map

Secret volume mount (representative YAML)

volumeMounts:
  - name: db-credentials
    mountPath: /etc/db-credentials
    readOnly: true
volumes:
  - name: db-credentials
    secret:
      secretName: db-credentials-secret

Create these objects from the command line:

kubectl create configmap app-config-map --from-file=./config/
kubectl create secret generic db-credentials-secret --from-literal=username=admin --from-literal=password=$(openssl rand -base64 12)

Sharp edges

• ConfigMap updates do not automatically restart pods. A change to a ConfigMap’s data will not be reflected in running containers until the pod is restarted (or the volume is remounted, which depends on the orchestrator’s implementation). Teams deploying config changes often wonder why the new values aren’t picked up.
• Secret volume mounts require a pod restart after the Secret is updated. Unlike ConfigMaps (where some runtimes support propagation), Secrets are mounted as files at pod start; a kubectl apply on the Secret object alone will not refresh the running pod.
• Environment variable precedence (env > configMap > secret) is a source of subtle bugs. If a developer defines an env var with the same name as a key in an mounted ConfigMap, the env var wins, and the mounted file appears empty or outdated. This ordering is orchestrator-specific but follows a de facto standard across most implementations.

Autoscaling Based on Load

Adding or removing pods in response to traffic is the pattern that keeps latency reasonable under variable demand. The Horizontal Pod Autoscaler (HPA) is the canonical mechanism.

Concrete command to create an HPA

kubectl autoscale deployment app --min=2 --max=10 --cpu-percent=75

The resulting HPA resource stores a target average utilization percentage. The orchestrator’s controller loop queries the metrics server every 15 seconds and adjusts the replica count accordingly.

Sharp edges

• Metrics-server latency (often 30–60 seconds) means the HPA reacts to load after it has already built up. During a sudden traffic spike, you may see pods added, but the new pods won’t be ready to serve traffic for another minute or two due to startup time and probe cooling-off periods.
• Burstable workloads with high CPU variance can confuse the cpu-percent metric. A pod that occasionally bursts to 200% CPU will be seen as “above target” even during normal operation, causing the HPA to scale up unnecessarily.
• External metrics (queue depth, request latency) require a custom metrics adapter. If the adapter is misconfigured, the HPA may report 0 or null for the metric, leading to either no scaling action or erratic scaling behavior that is difficult to diagnose without kubectl get hpa -o yaml and inspection of the currentMetrics field.

Summary Checklist

Pattern	Primary Command	Common Failure
Rolling update	kubectl set image … && kubectl rollout status	Probe misconfiguration stalls rollout
Canary/blue-green	kubectl patch service … or mesh weight config	Sticky sessions mask traffic split
Liveness/readiness	Pod spec httpGet probes	Readiness missing → 502s; liveness too aggressive → restart loop
Resource requests/limits	kubectl edit deployment	OOMKilled without graceful shutdown
Node drain/cordon/drain	kubectl cordon … && kubectl drain …	DaemonSets or taints block drainage
ConfigMap/Secret injection	kubectl create configmap/secret …	Updated ConfigMap/Secret not reflected until pod restart
HPA autoscaling	kubectl autoscale … --cpu-percent=75	Metrics-server lag or external metric adapter misconfiguration

These patterns form the operational vocabulary of any containerised workload running above a cluster manager. Memorise the commands, respect the probe boundaries, and verify that resource requests and limits match your actual workload profile—the most common production incidents in orchestrated environments trace back to one of the sharp edges listed above.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.