Cloud Capacity Planning | The Garnet Wiki

Cloud Capacity Planning is the practice of sizing and provisioning cloud infrastructure to meet expected and variable workloads—ensuring systems perform reliably under load while avoiding over-provisioning and under-utilization. It matters when a service begins to experience latency, timeouts, or increased error rates, or when scaling is reactive rather than proactive. It is the foundation of predictable performance, efficient resource use, and cost stability in production systems.

Assessing Workload Patterns

Workload patterns define the shape of demand: how load varies over time, by user, by region, or across services. The first step is to collect time-series metrics from compute, storage, and network layers.

Use curl to fetch metrics from a monitoring endpoint:

curl -H "Authorization: Bearer $API_TOKEN" \
     "https://monitoring.example.com/api/v1/metrics?metric=cpu_utilization&start=now-24h&step=5m"

Response is a JSON array of timestamped values:

[
  { "timestamp": "2023-10-05T14:00:00Z", "value": 0.72 },
  { "timestamp": "2023-10-05T14:05:00Z", "value": 0.81 },
  ...
]

Identify patterns by plotting these values. Sudden spikes correlate with scheduled jobs; gradual ramps indicate user growth. Seasonal patterns emerge over weeks or months.

A critical failure mode: under-sampling. If metrics are collected every 15 minutes but a job runs every 10 minutes, the peak during the job execution is missed. The system appears stable, but latency spikes occur every 10 minutes and go unnoticed.

Use kubectl to collect pod-level metrics:

kubectl top pods --all-namespaces --containers

This returns a table showing CPU and memory usage per container. A pod consuming 95% CPU consistently over 30 minutes signals a need for scaling.

When a service experiences high error rates, examine the latency distribution. Use pg_dump to export query latency data from a PostgreSQL instance:

pg_dump -Fc -t query_log -U app_user -h db.example.com -d app_db > query_log.dump

Import into a time-series database like InfluxDB using influx write:

influx write --bucket "app-latency" --org "engineering" --precision "ns" \
             --file query_log.dump

Now query for p95 latency over 1-hour windows:

from(bucket: "app-latency")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "query_latency")
  |> aggregateWindow(every: 1h, fn: quantile(q: 0.95), columns: ["_value"])

High p95 latency with low average suggests occasional long-running queries—potential bottlenecks.

Sizing Compute Resources

To size compute, determine the number of instances or containers needed to serve the expected load. Start with requests per second (RPS) and average response time.

Use the formula:

Required CPU cores = (RPS × Average response time in seconds) / (Available time per core)

For a service with 1,000 RPS and 500 ms average response time, and each core handles 1 second of work per second:

Required cores = (1000 × 0.5) / 1 = 500

Now map this to instance types. A single c5.4xlarge instance has 16 vCPUs and 64 GiB RAM. If each instance runs 10 worker processes, and each process consumes 500 MB memory, the instance supports 128 processes.

But sizing is not just about total CPU. Memory pressure causes swapping, which degrades performance silently. Monitor with free -h:

free -h

Output:

              total        used        free      shared  buff/cache   available
Mem:           62G         50G         10G        1.2G         20G         12G
Swap:          32G          8G         24G

When available memory drops below 15% of total, the system is under memory pressure. The Swap column shows active swapping. Use vmstat 1 to detect memory thrashing:

vmstat 1

Output:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0  8192000 10485760 20971520 20971520    0    0    40    60  150  200  12  8  70  10  0
 1  0  8192000  9437184 20971520 20971520    0    0    30    50  140  190  15  7  68  10  0

When si (swap-in) and so (swap-out) are non-zero and increasing, the system is swapping actively. Each page fault adds 1–5 ms latency.

Scale compute by horizontal scaling. Use kubectl scale to adjust replica count:

kubectl scale deployment/webapp --replicas=20

But scaling is not instant. A new pod takes 15–30 seconds to start. During this time, incoming requests are queued. Use kubectl rollout status to monitor rollout progress:

kubectl rollout status deployment/webapp

If the rollout times out, the service experiences a downtime spike. The readinessProbe must be tuned: if the probe fails, the pod is not ready, and traffic is not routed to it.

A common mistake: overlapping scaling decisions. When the system scales up, the new instances do not immediately absorb load. The initial load spike during scaling causes a capacity ripple. A service with 1,000 RPS and 100 new instances added over 30 seconds sees a 2,000 RPS spike as old instances handle backlog and new instances start.

Managing Storage and I/O Capacity

Storage capacity planning requires understanding throughput, latency, and IOPS. A single database query may require multiple I/O operations, each contributing to end-to-end latency.

Use iostat -x 1 to monitor disk I/O:

iostat -x 1

Output:

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           15.20    0.50    8.50   10.00    1.20   64.60

Device             r/s   w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda                 5.0   3.0    200.0    150.0    40.0     1.5      120.0   20.0   85.0

r/s and w/s: reads and writes per second.
rkB/s, wkB/s: throughput in kilobytes.
avgrq-sz: average request size in sectors (512 bytes).
await: average time to complete a request.
svctm: service time (time spent on the device).
%util: percentage of time the device was busy.

When await exceeds svctm, the queue is growing. When %util is above 80%, the disk is saturated.

A common failure: I/O bottleneck at the database layer. A query that executes in 100 ms locally takes 500 ms in production because the data is not in memory and disk I/O is the limiting factor.

Use pg_stat_statements to profile query performance:

SELECT query, calls, total_time, mean_time
FROM pg_stat_statements
ORDER BY total_time DESC
LIMIT 10;

A query consuming 150 ms per call with 1,000 calls per second contributes 150 seconds of work per second—equivalent to 150 CPU cores. If the database has 16 cores, it is overloaded.

To reduce I/O load, pre-warm the database cache. Use a script to execute common queries during off-peak hours:

#!/bin/bash

psql -U app_user -d app_db -c "
  SELECT * FROM users WHERE created_at > '2023-01-01';
  SELECT * FROM orders WHERE status = 'shipped';
  SELECT * FROM products ORDER BY price LIMIT 100;
"

Run this script during the nightly maintenance window. The data is now in the buffer pool, reducing I/O on the next morning peak.

Preparing for Peak Load

Peaks—daily, weekly, seasonal—require proactive planning. The key is to simulate peak load using real data.

Use k6 to run load tests:

k6 run --vus 1000 --duration 1h --output influxdb=host:influx.example.com:8086 \
       --thresholds http_req_duration=avg<500 \
       --thresholds http_req_failed=rate<0.02 \
       scripts/load-test.js

The script load-test.js defines the test:

import http from 'k6/http';
import { check, group } from 'k6';

export default function () {
  group('User login', () => {
    const res = http.post('https://api.example.com/auth/login', {
      email: '[email protected]',
      password: 'secret',
    });
    check(res, { 'login status is 200': (r) => r.status === 200 });
  });

  group('Fetch user profile', () => {
    const res = http.get('https://api.example.com/users/123');
    check(res, { 'profile load under 300ms': (r) => r.timings.duration < 300 });
  });
}

The test runs 1,000 virtual users for one hour. During execution, k6 emits metrics to InfluxDB, which can be visualized in Grafana.

A silent failure: inadequate test data. The test uses a fixed set of users, but in production, user data is distributed across multiple regions. The test does not reflect real-world distribution, leading to inaccurate load profiles.

Use canary deployment to validate capacity changes. Deploy a new version to 10% of traffic and monitor metrics.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp-canary
  labels:
    app: webapp
    version: 2.1
spec:
  replicas: 10
  selector:
    matchLabels:
      app: webapp
      version: 2.1
  template:
    metadata:
      labels:
        app: webapp
        version: 2.1
    spec:
      containers:
      - name: webapp
        image: registry.example.com/webapp:v2.1
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "1"
EOF

Then create a Service that routes traffic:

apiVersion: v1
kind: Service
metadata:
  name: webapp
spec:
  selector:
    app: webapp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
  type: LoadBalancer

Use kubectl patch to shift traffic:

kubectl patch service/webapp -p '
{
  "spec": {
    "selector": {
      "app": "webapp",
      "version": "2.1"
    }
  }
}'

But traffic shift is not atomic. During the patch, some requests are served by old pods, others by new. Use kubectl rollout status to confirm the canary is stable.

If metrics show increased latency or error rate, use kubectl describe to inspect the pods:

kubectl describe pod webapp-canary-7c9b8d4f5c

Look for ContainerStatus and Events sections. A pod stuck in ContainerCreating state with FailedScheduling event indicates insufficient CPU or memory in the cluster.

Automating Capacity Adjustments

Automate capacity planning with scaling policies. Define thresholds that trigger scaling actions.

Use kubectl autoscale to create a horizontal pod autoscaler:

kubectl autoscale deployment/webapp \
  --cpu-percent=70 \
  --min=5 \
  --max=100

This scales the deployment based on CPU usage. But the default CPU metric is container_cpu_usage_seconds_total. It is collected via cAdvisor, and the metric is sampled every 10 seconds.

A failure mode: metric latency. The metric is 10 seconds behind, so a sudden spike in CPU is not detected until 10 seconds after it occurs. The system responds too slowly.

Use kubectl get hpa to inspect the autoscaler:

kubectl get hpa

Output:

NAME       REFERENCE           TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
webapp     Deployment/webapp   65%/70%   5         100       20         2h

The TARGETS column shows current CPU utilization and the target. When the average CPU across all pods is 65%, the scaler is under target.

To improve responsiveness, use custom metrics. Define a metric from the application logs:

kubectl apply -f - <<EOF
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: webapp
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: webapp
  minReplicas: 5
  maxReplicas: 100
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "100"
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
EOF

The http_requests_per_second metric is collected via a Prometheus exporter in the application. It is available via a /metrics endpoint.

When the average number of requests per second across all pods is 100, the scaler increases replicas. The combination of CPU and request rate provides a more accurate picture than CPU alone.

When scaling fails, examine the autoscaler logs:

kubectl logs -l app=webapp-autoscaler

Look for entries like:

2023-10-05T14:32:15Z INFO: Scaling webapp from 20 to 25 replicas due to HTTP request rate: 120.0 (target: 100.0)
2023-10-05T14:32:20Z ERROR: Cannot scale webapp: Failed to update deployment: Timeout waiting for deployment rollout

A timeout during deployment rollout means the service is not ready to handle traffic. The autoscaler may have scaled too aggressively, but the new pods were not ready.

Monitoring and Iterating

Capacity planning is not a one-time task. It requires continuous monitoring and iteration.

Use watch to monitor key metrics:

watch -n 10 'curl -s http://monitoring.example.com/api/v1/metrics?metric=error_rate&last=1h'

This runs every 10 seconds, showing the error rate over the last hour. A rising error rate signals under-capacity.

Use journalctl to inspect service logs:

journalctl -u webapp.service --since "2023-10-05 14:00:00" --follow

This shows real-time logs from the webapp service. A spike in ERROR messages indicates a bottleneck.

When a service fails, use strace to trace system calls:

strace -f -o /tmp/webapp-trace.log -p $(pgrep webapp)

This traces all system calls made by the webapp process and its children. Look for epoll_wait, read, write, and open calls that are taking longer than expected.

The pattern is clear: capacity planning is a feedback loop. Metrics inform sizing. Sizing enables better performance. Performance generates new metrics. The loop closes, and the system evolves.

This page was rewritten on 16 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.