Multi Cloud Cost Comparison

Orientation

Multi-cloud cost comparison is the practice of extracting, normalizing, and contrasting spend data across two or more cloud providers. It matters when engineering organizations run workloads across AWS, Azure, GCP, or other providers, when planning migrations, evaluating vendor diversification, allocating shared budgets, or reconciling finance and engineering ownership. Unlike single-cloud spend analysis, this work must reconcile different billing schemas, tagging conventions, currency systems, and API surface areas. The following sections are organized around the specific tasks a reader may want to perform, with concrete commands and the sharp edges that only appear in production.

Extract cost data from each cloud

Most cloud providers expose cost data through a REST API that requires bearer-token authentication and accepts a date range. A typical export request with curl follows this pattern:

curl -sSf -G https://billing-provider.example.com/v1/export \
  --header "Authorization: Bearer $TOKEN" \
  --data-urlencode "start=2024-01-01" \
  --data-urlencode "end=2024-01-31" \
  -o costs_raw.json

The -G flag sends data as query parameters, --data-urlencode safely encodes the date range, and -o writes the JSON response to disk. Replace the host, token, and query parameters for your specific provider; the curl flags and overall structure are standard across HTTP-based billing interfaces.

If your organization funnels billing data into a data warehouse, you can stream the export directly into a load pipeline:

curl -sSf -G https://billing-provider.example.com/v1/export \
  --header "Authorization: Bearer $TOKEN" \
  --data-urlencode "start=2024-01-01" \
  --data-urlencode "end=2024-01-31" | psql -c "COPY staging_costs FROM STDIN (FORMAT csv)"

staging_costs is a temporary table into which raw rows are bulk-loaded before schema normalization. The COPY … FROM STDIN syntax is portable across psql installations and avoids intermediate file creation.

Align data into a comparable schema

Cloud providers return cost line items with different field names, nested structures, and optional columns. A common approach is to map each provider’s response into a shared relational schema before further analysis. The following psql schema creates a normalized table that can hold cost data from any provider:

CREATE TABLE IF NOT EXISTS cloud_costs (
  provider TEXT NOT NULL,
  account_id TEXT,
  invoice_id TEXT,
  service TEXT,
  usage_type TEXT,
  amount NUMERIC(12,2),
  currency TEXT,
  timestamp DATE,
  region TEXT,
  tags JSONB
);

Key decisions in this design:

• provider is a free-text tag (e.g., aws, azure, gcp) rather than a constrained enum, because provider identifiers may shift across API versions.
• tags is stored as JSONB to accommodate the wildly different tagging models each cloud uses, without forcing a rigid column layout that would require migration every time a new tag key appears.
• amount and currency are kept separate so conversion logic can be applied later, or reports can be filtered by a single currency.

Load the raw export into a staging table first, then transform:

CREATE TEMP TABLE staging_costs (
  raw JSONB
);

COPY staging_costs FROM '/path/to/costs_raw.json' (FORMAT json);

INSERT INTO cloud_costs (provider, account_id, invoice_id, service, usage_type, amount, currency, timestamp, region, tags)
SELECT
  (raw->>'provider')::TEXT AS provider,
  (raw->>'accountId')::TEXT AS account_id,
  (raw->>'invoiceId')::TEXT AS invoice_id,
  (raw->>'service')::TEXT AS service,
  (raw->>'usageType')::TEXT AS usage_type,
  (raw->>'amount'->>'numeric')::NUMERIC(12,2) AS amount,
  (raw->>'currency')::TEXT AS currency,
  TO_DATE((raw->>'date')::TEXT, 'YYYY-MM-DD') AS timestamp,
  (raw->>'region')::TEXT AS region,
  raw->>'tags' AS tags
FROM staging_costs
WHERE raw->>'provider' IS NOT NULL;

The ->> operator extracts JSON values as text, and the ::TYPE casts are standard psql syntax. If a provider omits a field (e.g., usage_type), the column will be NULL, which is intentional: the normalized schema tolerates missing data rather than rejecting it.

Query and compare across providers

Once cost data from multiple clouds reside in the same table, comparison queries become straightforward. The following example aggregates spend by provider and service for a given month, grouping by currency so you can see where conversion is needed:

SELECT provider,
       service,
       currency,
       SUM(amount) AS total_amount
FROM cloud_costs
WHERE timestamp >= '2024-01-01' AND timestamp < '2024-02-01'
GROUP BY provider, service, currency
ORDER BY provider, total_amount DESC;

If you need a single-currency view, wrap the query with a conversion layer. A common pattern is to maintain a small reference table of daily FX rates and join it in the query, but the specifics of exchange-rate ingestion are outside the scope of this page.

To surface the largest divergences between providers for the same logical workload, you can use a FULL OUTER JOIN pattern on a normalized service identifier:

WITH per_provider AS (
  SELECT provider,
         service,
         SUM(amount) AS spend
  FROM cloud_costs
  WHERE timestamp >= '2024-01-01' AND timestamp < '2024-02-01'
  GROUP BY provider, service
)
SELECT COALESCE(a.provider, b.provider) AS provider,
       COALESCE(a.service, b.service) AS service,
       a.spend AS aws_spend,
       b.spend AS azure_spend
FROM per_provider a
FULL OUTER JOIN per_provider b
  ON a.service = b.service
 AND a.provider = 'aws'
 AND b.provider = 'azure'
WHERE COALESCE(a.spend, 0) > 0 OR COALESCE(b.spend, 0) > 0
ORDER BY total_spend DESC;

This query returns a row for every service that appears in at least one provider, filling NULL where a service is absent from one cloud. The result set makes it immediately obvious which services have no direct counterpart across your multi-cloud footprint.

Automate and schedule

Comparisons are only useful when they run regularly. A typical automation consists of three steps: fetch fresh billing data, load it into the normalized schema, and re-run comparison queries. Here’s a minimal shell script that orchestrates the cycle:

#!/usr/bin/env bash
set -euo pipefail


curl -sSf -G https://billing-aws.example.com/v1/export \
  --header "Authorization: Bearer $AWS_TOKEN" \
  --data-urlencode "start=$(date -d 'last month' +%Y-%m-01)" \
  --data-urlencode "end=$(date -d 'last month' +%Y-%m-31)" \
  -o /tmp/costs_aws.json

curl -sSf -G https://billing-azure.example.com/v1/export \
  --header "Authorization: Bearer $AZURE_TOKEN" \
  --data-urlencode "start=$(date -d 'last month' +%Y-%m-01)" \
  --data-urlencode "end=$(date -d 'last month' +%Y-%m-31)" \
  -o /tmp/costs_azure.json

# 2. Load into the normalized schema (psql must have access to the DB)
psql -U finops_user -d cost_warehouse -c "
  COPY staging_costs FROM STDIN (FORMAT json);
  INSERT INTO cloud_costs (provider, account_id, invoice_id, service, usage_type, amount, currency, timestamp, region, tags)
  SELECT 'aws'::TEXT, (raw->>'accountId')::TEXT, (raw->>'invoiceId')::TEXT,
         (raw->>'service')::TEXT, (raw->>'usageType')::TEXT,
         (raw->>'amount'->>'numeric')::NUMERIC(12,2),
         (raw->>'currency')::TEXT,
         TO_DATE((raw->>'date')::TEXT, 'YYYY-MM-DD'),
         (raw->>'region')::TEXT,
         raw->>'tags'
  FROM staging_costs
  WHERE (raw->>'provider')::TEXT = 'aws';
" < /tmp/costs_aws.json

psql -U finops_user -d cost_warehouse -c "
  COPY staging_costs FROM STDIN (FORMAT json);
  INSERT INTO cloud_costs (provider, account_id, invoice_id, service, usage_type, amount, currency, timestamp, region, tags)
  SELECT 'azure'::TEXT, (raw->>'accountId')::TEXT, (raw->>'invoiceId')::TEXT,
         (raw->>'service')::TEXT, (raw->>'usageType')::TEXT,
         (raw->>'amount'->>'numeric')::NUMERIC(12,2),
         (raw->>'currency')::TEXT,
         TO_DATE((raw->>'date')::TEXT, 'YYYY-MM-DD'),
         (raw->>'region')::TEXT,
         raw->>'tags'
  FROM staging_costs
  WHERE (raw->>'provider')::TEXT = 'azure';
" < /tmp/costs_azure.json

# 3. Run comparison and output a markdown report
psql -U finops_user -d cost_warehouse -c "\copy (SELECT * FROM compare_monthly_spend()) TO STDOUT WITH (FORMAT csv)" > /tmp/comparison_report.csv

Make the script executable and schedule it with cron:

crontab -e
# Add the following line to run comparison on the first day of every month at 03:00 UTC
0 3 1 * * /usr/local/bin/cost-compare.sh >> /var/log/cost-compare.log 2>&1

The cron entry uses standard crontab syntax; the script path and user depend on your environment. All commands (curl, psql, date) are universally available on most Linux-based systems.

Sharp edges — what silently fails

• Tagging inconsistency: One provider may tag a resource as Environment=prod, another as env:prod, and a third may omit tags entirely. Without a pre‑ingest mapping step, these line items are treated as unrelated, inflating the apparent service count and obscuring true spend.
• Currency conversion gaps: Amounts are returned in the provider’s local currency. If you compare raw amount values across clouds, the numbers are meaningless without conversion. A missing or stale FX rate table will silently produce totals that appear accurate but are actually distorted.
• Free‑tier inclusion: Some services include a monthly free quota (e.g., 1 GB of storage, 1 million API calls). Raw cost exports often include the full invoice amount before quota subtraction. Comparing unadjusted figures makes a “free” service look expensive in one cloud and cheap in another.
• Region mapping mismatches: us-east-1 on AWS corresponds to East US on Azure and us-east1 on GCP. Without an explicit mapping table, grouping by the raw region string produces false splits or missed aggregations, especially when services are region‑scoped.
• API pagination and silent data loss: Billing APIs frequently paginate results using nextToken or pageToken. If a request window spans many months or high‑volume accounts, a naïve curl call that ignores pagination will only retrieve the first page, dropping line items and yielding under‑reported totals. Production pipelines must loop until nextToken is absent.
• Timezone alignment: Invoice periods may start on the first day of a calendar month in one provider and on a custom billing cycle in another. Querying “last month” with timestamp >= '2024-01-01' AND timestamp < '2024-02-01' assumes UTC‑midnight boundaries. If a provider stores timestamps in local time without timezone metadata, rows can fall outside the intended window, causing quiet mismatches between expected and actual row counts.
• Rate‑limit throttling: Many billing APIs enforce per‑second or per‑minute request limits. A script that fires concurrent curl calls across multiple providers may hit a 429 response. If the script exits on error without retry logic, the current month’s export will be incomplete, and the comparison report will be based on partial data.

Each of these failure modes is learned after a comparison report is generated and the numbers don’t reconcile with finance’s general‑ledger totals. Building idempotent fetch loops, maintaining a tag‑mapping reference table, and normalizing timestamps to UTC before loading into the warehouse are the standard mitigations.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.