Systematic cloud cost optimization for AWS, Azure, and GCP. Covers rightsizing, reserved capacity, spot instances, storage tiering, and building a FinOps practice.
Cloud cost optimization is not a project; it is a continuous, cross-functional discipline. This playbook provides a structured, repeatable approach to identifying, implementing, and sustaining cloud cost savings across engineering, operations, and finance teams. It is the operating manual for reducing cloud spend without sacrificing performance, reliability, or velocity—especially when cloud bills grow faster than teams can react.
Before optimizing, teams must understand their current state. This begins with a clear, unified view of all cloud resources and their associated costs.
Cloud providers export billing data in raw, often inconsistent formats. Use curl to retrieve daily cost and usage reports (CUR) from AWS S3, Azure Blob Storage, or GCP Cloud Storage.
curl -X GET \
-H "Authorization: Bearer $AWS_ACCESS_TOKEN" \
-H "Accept: application/json" \
https://s3.amazonaws.com/my-billing-bucket/cur/2024-04-01.parquetThe response includes a cost, usage, and resourceId field, but the resourceId format varies: AWS uses arn:aws:ec2:us-east-1:123456789012:instance/i-12345678, while GCP uses projects/my-project/zones/us-central1-a/instances/my-instance. Normalize these into a common key: cloud_id.
Create a costs table in a data warehouse using psql:
CREATE TABLE costs (
cloud_id TEXT PRIMARY KEY,
billing_period DATE,
cost DECIMAL(12,4),
usage_hours DECIMAL(8,2),
resource_type TEXT,
region TEXT,
tags JSONB,
provider TEXT
);Load the data via pg_bulkload or COPY with FORMAT PARQUET.
Tagging is the backbone of cost visibility. Use kubectl to manage a central tag policy across Kubernetes clusters:
apiVersion: v1
kind: ConfigMap
metadata:
name: cost-allocation-tags
namespace: finops
data:
default-tags: |
{
"Environment": "Production",
"Owner": "Engineering",
"Project": "User-Service",
"Team": "Identity",
"Service": "User-API"
}
required-tags: |
[
"Environment",
"Owner",
"Project",
"Service"
]Apply with:
kubectl apply -f configmap.yamlA missing Environment tag triggers a cost anomaly. Use aws cloudtrail to record tag changes:
{
"eventSource": "ec2.amazonaws.com",
"eventName": "CreateTags",
"responseElements": {
"return": true,
"tagSet": [
{
"key": "Environment",
"value": "Staging"
}
]
}
}Failure mode: a tag is applied to a resource but not propagated to its parent (e.g., EC2 instance tagged but not the Auto Scaling Group). Use jq to extract and flatten tags:
aws ec2 describe-instances --query 'Reservations[].Instances[].{InstanceId:InstanceId, Tags:Tags}' \
| jq -r '.[] | select(.Tags | length > 0) | .InstanceId as $id | .Tags[] | [.key, .value] | @tsv'With data in place, teams can systematically uncover cost inefficiencies.
Use kubectl to list all pods and their CPU and memory usage from Prometheus metrics:
kubectl get pods -n user-service -o wide
kubectl top pods -n user-service --containersExport metrics to a cost-performance table:
CREATE TABLE cost_performance (
pod_name TEXT,
namespace TEXT,
cpu_usage DECIMAL(6,2),
memory_usage DECIMAL(8,2),
cpu_requests DECIMAL(6,2),
memory_requests DECIMAL(8,2),
cost_per_hour DECIMAL(6,4),
utilization_ratio DECIMAL(5,3)
);A pod with cpu_usage < 0.2 and memory_usage < 0.3 is underutilized. Query for all such pods:
SELECT pod_name, namespace, cost_per_hour
FROM cost_performance
WHERE cpu_usage < 0.2 AND memory_usage < 0.3
ORDER BY cost_per_hour DESC;Use aws ec2 describe-instances to find EC2 instances with Status == "stopped" and LaunchTime older than 14 days:
aws ec2 describe-instances \
--filters "Name=instance-state-name,Values=stopped" \
--query 'Reservations[].Instances[?LaunchTime < `2024-03-15T00:00:00Z`].[InstanceId, LaunchTime, InstanceType]'For each instance, check if it has a Schedule tag. If not, it’s a candidate for automation.
Use gcloud compute instances list to identify GCP VMs with status == "TERMINATED" and creationTimestamp < "2024-03-15".
List EBS volumes with IOPS and ProvisionedThroughput values:
aws ec2 describe-volumes \
--filters "Name=volume-type,Values=gp2" \
--query 'Volumes[].{VolumeId:VolumeId, Size:Size, IOPS:Iops, Throughput:ProvisionedThroughput}'A volume is overprovisioned if IOPS > 2 * Size or Throughput > 100 MB/s. Use psql to flag such volumes:
SELECT VolumeId, Size, IOPS, Throughput,
CASE
WHEN IOPS > (2 * Size) THEN 'High IOPS'
WHEN Throughput > 100 THEN 'High Throughput'
ELSE 'Standard'
END AS overprovisioned_reason
FROM ebs_volumes
WHERE IOPS > (2 * Size) OR Throughput > 100;Once opportunities are identified, teams execute with precision and consistency.
For EC2 instances, use aws ec2 modify-instance-attribute to update instance type based on usage patterns:
aws ec2 modify-instance-attribute \
--instance-id i-12345678 \
--instance-type t3.smallUse kubectl patch to update Kubernetes deployments:
apiVersion: apps/v1
kind: Deployment
metadata:
name: user-api
spec:
template:
spec:
containers:
- name: user-api
resources:
requests:
cpu: "0.5"
memory: "512Mi"
limits:
cpu: "1"
memory: "1Gi"Apply with:
kubectl patch deployment user-api -p "$(cat deployment-patch.yaml)"A common failure: requests and limits are mismatched. A pod with requests.cpu = 0.5 but limits.cpu = 1 is often underutilized and overpriced. Use kubectl describe pod to verify.
Define a lifecycle policy using curl to interact with a cloud-native automation service:
curl -X POST \
-H "Content-Type: application/json" \
-d '{
"name": "Auto-stop-staging",
"target": "ec2",
"schedule": "0 18 * * 1-5",
"actions": [
{
"type": "stop",
"condition": {
"tag": "Environment",
"value": "Staging"
}
},
{
"type": "start",
"condition": {
"tag": "Environment",
"value": "Production"
}
}
]
}' \
https://api.automation.cloud/automation-policiesA failure mode: a scheduled job runs but fails to update the LastUpdated timestamp in the database. Use systemctl to monitor a background service:
systemctl status cost-automation-worker
journalctl -u cost-automation-worker -fCheck for Error: Failed to connect to database or HTTP 500: Internal Server Error in the logs.
For Kubernetes, define HorizontalPodAutoscaler with a metric-based policy:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: user-api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: user-api
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "20"Apply with:
kubectl apply -f hpa.yamlA silent failure: the http_requests_per_second metric is not collected by Prometheus. Use kubectl get hpa and kubectl describe hpa user-api-hpa to diagnose.
For S3, use aws s3api put-bucket-lifecycle to define lifecycle rules:
{
"Rules": [
{
"ID": "move-to-glacier",
"Status": "Enabled",
"Prefix": "logs/",
"ExpirationInDays": 365,
"Transitions": [
{
"Days": 90,
"StorageClass": "STANDARD_IA"
},
{
"Days": 180,
"StorageClass": "GLACIER"
}
]
}
]
}Apply with:
aws s3api put-bucket-lifecycle \
--bucket my-billing-data \
--lifecycle-configuration file://lifecycle.jsonA common mistake: Transition rules are applied in order, but the Days count starts from the bucket creation, not from the last transition. Use aws s3api get-bucket-lifecycle to verify.
Optimization is not a one-time event. Teams must embed it into their rhythms.
Use a cost-review script that generates a summary:
#!/bin/bash
# cost-review.sh
echo "Cloud Cost Review: $(date)"
echo "===================="
echo "Top 5 Cost Drivers:"
psql -c "SELECT resource_type, SUM(cost) FROM costs GROUP BY resource_type ORDER BY sum DESC LIMIT 5" -t
echo "Underutilized Resources:"
psql -c "SELECT pod_name, cpu_usage, memory_usage, cost_per_hour FROM cost_performance WHERE cpu_usage < 0.2 AND memory_usage < 0.3" -t
echo "Pending Optimization Tasks:"
psql -c "SELECT id, description, due_date FROM optimization_tasks WHERE status = 'pending'" -t
echo "Automated Actions Executed:"
systemctl status cost-automation-worker --no-pager -l | tail -n 10Run via cron:
0 9 * * 1 /opt/scripts/cost-review.sh | mail -s "Weekly Cloud Cost Review" [email protected]Maintain a optimization_tasks table:
CREATE TABLE optimization_tasks (
id SERIAL PRIMARY KEY,
description TEXT NOT NULL,
resource_id TEXT,
cost_savings DECIMAL(10,2),
status TEXT CHECK (status IN ('pending', 'in-progress', 'completed', 'failed')),
due_date DATE,
executed_at TIMESTAMP,
notes TEXT
);Update the status via curl:
curl -X PUT \
-H "Content-Type: application/json" \
-d '{"status": "completed", "executed_at": "2024-04-10T14:30:00Z", "notes": "Instance resized from t3.large to t3.small"}' \
https://api.cost-ops.org/tasks/123A failure: the executed_at timestamp is in UTC, but the team expects local time. Use psql to convert:
SELECT id, description, executed_at AT TIME ZONE 'America/New_York' AS executed_at_ny
FROM optimization_tasks
WHERE status = 'completed';When a cost spike occurs, teams use the playbook to triage.
Use pgbench to simulate a cost anomaly detection pipeline:
CREATE TABLE cost_anomalies (
id SERIAL PRIMARY KEY,
billing_period DATE,
expected_cost DECIMAL(12,4),
actual_cost DECIMAL(12,4),
anomaly_score DECIMAL(5,3),
detected_at TIMESTAMP,
severity TEXT
);
-- Insert anomaly detection logic
INSERT INTO cost_anomalies (billing_period, expected_cost, actual_cost, anomaly_score, detected_at)
SELECT
billing_period,
AVG(cost) * 1.2 AS expected_cost,
SUM(cost) AS actual_cost,
(SUM(cost) - AVG(cost) * 1.2) / AVG(cost) AS anomaly_score,
NOW() AS detected_at
FROM costs
WHERE billing_period >= (NOW() - INTERVAL '7 days')
GROUP BY billing_period
HAVING (SUM(cost) - AVG(cost) * 1.2) / AVG(cost) > 0.1;A silent failure: anomaly_score is calculated but the severity is not updated. Use psql to classify:
UPDATE cost_anomalies
SET severity = CASE
WHEN anomaly_score > 0.3 THEN 'Critical'
WHEN anomaly_score > 0.15 THEN 'High'
ELSE 'Medium'
END
WHERE severity IS NULL;Use systemctl to trigger a notification service:
systemctl restart cost-anomaly-notifierThis playbook is not a static document. It is a living system. Teams execute it weekly, refine it monthly, and adapt it to new services, platforms, and patterns. The cost of cloud is not just a number—it is a practice.
This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy