Canary Deployments

Canary deployments gradually roll out a new application version to a subset of users or infrastructure before full release. This technique reduces risk during releases by limiting the impact of failures and enabling quick rollback.

Deploying to a Subset of Traffic

To route traffic to a canary deployment, configure your load balancer or ingress controller to split requests between old and new versions. For Kubernetes deployments, update the service selector to include pods running both versions:

apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
    version: v1
  ports:
    - port: 80
      targetPort: 8080

Use kubectl patch to modify the service selector:

kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"v2"}}}'

The service will now route traffic to pods labeled with version=v2. Update both labels and selectors simultaneously to avoid traffic disruption.

Traffic Splitting Configuration

Configure traffic splitting using HTTP headers or query parameters. For example, when using NGINX Ingress Controller, define a custom annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp-service
            port:
              number: 80

Set canary-weight to 10 for 10% of traffic. Update the annotation value to adjust split percentage:

kubectl patch ingress myapp-ingress -p '{"metadata":{"annotations":{"nginx.ingress.kubernetes.io/canary-weight":"25"}}}'

Canary Rollout Automation

Automate progressive traffic shifts using Kubernetes rollout strategies. Create a new deployment with updated image:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-canary
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
      version: v2
  template:
    metadata:
      labels:
        app: myapp
        version: v2
    spec:
      containers:
      - name: myapp
        image: myapp:v2.0

Apply with kubectl apply -f deployment-canary.yaml. Gradually increase replica count or weight.

Monitoring and Rollback

Monitor application health using liveness and readiness probes. Add a readiness probe to detect when canary pods are ready:

spec:
  template:
    spec:
      containers:
      - name: myapp
        image: myapp:v2.0
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10

Check pod status with:

kubectl get pods -l version=v2

When a canary deployment fails, rollback using:

kubectl rollout undo deployment/myapp-canary

Health Check Integration

Implement custom health checks in your application code to signal readiness. Expose an endpoint like /healthz that returns HTTP 200 when service is ready.

Configure the readiness probe to call this endpoint:

readinessProbe:
  httpGet:
    path: /healthz
    port: 8080

If the health check fails, the pod becomes unavailable for traffic. A failed readiness probe does not immediately terminate the pod but prevents it from receiving requests.

Traffic Weight Management

Manage traffic distribution using weighted routing. In Kubernetes, use a service selector to target specific versions:

apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
    version: v2
  ports:
    - port: 80

Update the selector to shift traffic:

kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"v1"}}}'

Gradual Rollout

Progressively increase traffic to canary deployments. Start with 5% and move to 25%, then 50%, and finally 100%.

Use kubectl scale to modify replica counts:

kubectl scale deployment/myapp-canary --replicas=3

Each increase should be followed by health checks.

Configuration Management

Store rollout parameters in a configuration file or secret. Example configuration for traffic split:

apiVersion: v1
kind: ConfigMap
metadata:
  name: canary-config
data:
  traffic-weight: "10"
  timeout-seconds: "300"

Mount the config map into pods:

spec:
  containers:
  - name: myapp
    image: myapp:v2.0
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config

Rollback Process

When issues occur, rollback to previous version using:

kubectl rollout undo deployment/myapp-canary

Check deployment status with:

kubectl rollout status deployment/myapp-canary

Ensure all replicas are running before proceeding.

Canary Deployment Lifecycle

Start a new deployment by applying updated manifests:

kubectl apply -f deployment-canary.yaml

Monitor progress with:

kubectl get deployments
kubectl get pods

When ready, shift traffic gradually. Complete the rollout by updating service selectors or annotations.

Error Handling

If a canary pod fails health checks, it will not receive traffic. Monitor logs with:

kubectl logs -l version=v2

Check for common issues like missing environment variables, database connection failures, or misconfigured readiness probes.

Version Labeling Strategy

Use consistent version labeling to distinguish between production and canary deployments. For example:

• Production: version: v1
• Canary: version: v2-canary

This allows filtering and targeting specific versions.

Traffic Control Commands

Update traffic routing with direct service modifications:

kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"v2"}}}'

Adjust weight configuration:

kubectl patch ingress myapp-ingress -p '{"metadata":{"annotations":{"nginx.ingress.kubernetes.io/canary-weight":"50"}}}'

Monitoring Signals

Monitor key metrics during canary deployment:

• Pod readiness status
• Response time and error rates
• Resource utilization (CPU, memory)
• Application health endpoint responses

Use kubectl top pods to check resource usage.

Canary Deployment Validation

Validate that traffic is properly routed by testing specific endpoints. For example, send requests to /healthz and verify responses from expected versions.

Monitor service logs to ensure proper routing:

kubectl logs -l app=myapp -l version=v2

Traffic Splitting Patterns

Implement multiple traffic split strategies:

1. Percentage-based: Route 10% of traffic to canary
2. Header-based: Route based on request headers
3. Cookie-based: Route based on cookie values

Each pattern requires different configuration in the ingress or service layer.

Rollout Failure Recovery

If a rollout fails, revert immediately using:

kubectl rollout undo deployment/myapp-canary

Ensure rollback preserves all previous configuration and data.

Deployment Cleanup

Remove canary resources after successful rollout:

kubectl delete deployment myapp-canary
kubectl delete service myapp-canary-service

Verify cleanup with:

kubectl get deployments
kubectl get services

Common Pitfalls

1. Incorrect Service Selector

A mismatch between service selector and pod labels prevents traffic routing. Verify selectors match labels:

kubectl get pods -l version=v2 -o wide
kubectl get svc myapp-service -o yaml

2. Misconfigured Readiness Probes

An incorrect readiness probe causes pods to appear healthy before they're ready, leading to traffic routing issues.

3. Inconsistent Version Labels

Inconsistent labeling prevents proper targeting of specific deployment versions.

4. Missing Health Checks

Without proper health checks, failed canary deployments may continue receiving traffic.

Advanced Traffic Management

Use multiple service selectors for complex routing:

apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
    version: v2
  ports:
    - port: 80

Update multiple labels simultaneously to maintain consistency.

Rollout Progression

Gradually increase traffic using a step-by-step approach:

1. 5% traffic to canary
2. 10% traffic to canary
3. 25% traffic to canary
4. 50% traffic to canary
5. 100% traffic to canary

Each step requires verification of health metrics and application behavior.

Deployment Verification

Verify deployment completion with:

kubectl rollout status deployment/myapp-canary
kubectl get pods -l version=v2

Check service configuration matches expected routing rules.

Configuration Validation

Validate configuration files before applying:

kubectl apply --dry-run=client -f deployment-canary.yaml

This prevents accidental misconfiguration during deployment.

Canary Testing Tools

Use standard tools for canary testing:

• kubectl for deployment management
• HTTP clients for endpoint validation
• Logs inspection for health signals

No vendor-specific testing frameworks are required.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.