How to Manage Multi-Cloud Architecture

Multi-cloud architecture is essential when an organisation uses two or more cloud providers—AWS, Azure, GCP, and others—without relying on any single platform as the primary infrastructure layer. This approach is warranted when cost optimization, avoiding vendor lock-in, leveraging provider-specific capabilities, or meeting regulatory requirements across geographies. Management becomes critical when services span clouds, data flows between platforms, and operations span multiple CLI and API surfaces.

Define Cloud-Agnostic Service Boundaries

Define service boundaries using domain-driven design principles, where each service owns its data and APIs, and operates independently across cloud platforms. Services communicate via standardized contracts, typically REST over HTTP, and expose only well-documented, versioned APIs.

Each service must define its own deployment lifecycle, with infrastructure as code (IaC) that can be applied to any cloud. The IaC must be cloud-agnostic: it uses common primitives like virtual machines, load balancers, and databases, and abstracts provider-specific features behind a consistent interface.

For example, a service named user-profile deployed across AWS and GCP uses the same terraform configuration file with minimal overrides:

module "user_profile" {
  source = "./modules/user-profile"

  # Cloud-agnostic variables
  instance_type = "t3.medium"
  database_engine = "postgresql"
  region = "us-central1"
  environment = "production"
  memory_mb = 2048
  cpu_count = 2
}

The terraform configuration is applied uniformly across providers. A terraform apply command runs on AWS, and the same configuration deploys the service on GCP with only provider block adjustments:

# terraform/production/terraform.tfvars
provider "aws" {
  region = "us-east-1"
}

provider "google" {
  project = "my-company-prod"
  region  = "us-central1"
}

When a service fails to deploy, check the terraform plan output for drift. A common failure: a terraform apply on GCP misses a google_storage_bucket resource because the bucket variable was not defined in terraform.tfvars, despite being declared in the module. This occurs silently when the bucket variable has a default but no explicit value in the main configuration.

Unify Identity and Access Across Clouds

Identity and access management (IAM) must be consistent across all clouds. Use a centralized identity provider (IdP) such as Keycloak, Okta, or Auth0, and enforce identity federation using SAML or OIDC.

Each cloud platform must trust the IdP to issue JWT or SAML assertions. Services across clouds use the same authentication flow. For example, a user logs into a web app hosted on AWS, and their session is validated against a single IdP that also controls access to GCP-hosted analytics dashboards.

Configure identity federation using a saml2 metadata document. The IdP publishes a metadata file at https://idp.company.com/saml/metadata.xml, which the cloud providers consume.

On AWS, configure an IAM identity provider (IdP) using the aws iam create-saml-provider command:

aws iam create-saml-provider \
  --saml-metadata-document file://saml-metadata.xml \
  --name "Company-IdP"

On GCP, register the same IdP via the gcloud CLI:

gcloud iam saml-configs create company-idp \
  --saml-metadata-file=saml-metadata.xml \
  --display-name="Company IdP"

A common failure: the audience claim in the SAML assertion does not match the EntityID in the IdP metadata. The service returns 403 Forbidden with a SAML Assertion Validation Error: Audience mismatch when the aud claim in the JWT does not match the expected value.

Use curl to validate the SAML flow:

curl -X POST \
  -H "Content-Type: application/saml+xml" \
  -d @saml-request.xml \
  https://idp.company.com/saml/SSO

The response is a 302 redirect to the service with a SAMLResponse query parameter. The service extracts the SAMLResponse, validates the signature, and checks the Audience, Issuer, and Subject fields.

Centralize Configuration and Secrets

Centralize configuration and secrets in a shared, version-controlled system. Use a configuration store like Consul, etcd, or a simple JSON file in a Git repository.

Each cloud platform consumes configuration via a common interface. For example, a service reads configuration from config/production.json in a Git repository. The service runs git pull on startup and watches for changes.

Use a config service that exposes a REST API:

GET /config/v1/user-profile?env=production&region=us-east-1
{
  "database": {
    "host": "user-profile-prod-db.us-east-1.rds.amazonaws.com",
    "port": 5432,
    "name": "user_profile_prod"
  },
  "cache": {
    "host": "redis-cluster-gcp-1.internal",
    "port": 6379
  },
  "features": {
    "email_notifications": true,
    "profile_picture_upload": false
  }
}

To update configuration, commit to Git:

git add config/production.json
git commit -m "Enable profile picture uploads in US regions"
git push origin main

A deployment script watches for changes and reloads configuration:

#!/bin/bash
# deploy.sh

while true; do
  git fetch origin
  if git diff --quiet origin/main HEAD; then
    echo "Configuration changed, reloading..."
    curl -X POST http://config-service:8080/reload
    sleep 10
  else
    sleep 30
  fi
done

A silent failure: the config-service returns 200 OK but the service does not apply the new configuration. The issue is a mismatch between the config-version header and the ETag of the config file. The service checks the ETag but does not update the config-version in the response.

Standardize Monitoring and Observability

Standardize observability across clouds using a unified logging, metrics, and tracing stack. Use OpenTelemetry for telemetry collection, Prometheus for metrics, and Jaeger or OpenTelemetry Collector for distributed tracing.

Each service emits logs in JSON format with a standard schema:

{
  "timestamp": "2025-04-05T12:34:56.789Z",
  "level": "info",
  "service": "user-profile",
  "trace_id": "a1b2c3d4e5f6",
  "span_id": "f6e5d4c3b2a1",
  "message": "User profile updated successfully",
  "user_id": 12345,
  "duration_ms": 124
}

Deploy a fluentd agent to each host (VM or container) that forwards logs to a central elasticsearch cluster.

In fluentd, configure a forward input and output:

<system>
  <log_level>info</log_level>
</system>

<source>
  @type forward
  bind 0.0.0.0
  port 24224
</source>

<match **>
  @type copy
  <store>
    @type file
    path /var/log/fluentd
    format json
  </store>
  <store>
    @type forward
    host fluentd-collector
    port 24224
  </store>
</match>

Use prometheus to scrape metrics from services. Define a scrape_config in prometheus.yml:

scrape_configs:
  - job_name: 'user-profile'
    static_configs:
      - targets: ['user-profile-aws:9090', 'user-profile-gcp:9090']
    metrics_path: '/metrics'
    scheme: 'http'
    relabel_configs:
      - source_labels: [__address__]
        target_label: instance

A failure: a service reports 500 Internal Server Error but the logs show no error message. The issue is that the service logs only error level messages, but the prometheus metrics endpoint returns 200 OK even when the service is unresponsive.

Use curl to probe the metrics endpoint:

curl -v http://user-profile-aws:9090/metrics

The response is 200 OK, but the body contains no metrics. The service is not exporting metrics via prometheus_client. A common fix: add prometheus_client to the service and register the metrics endpoint.

Orchestrate Cross-Cloud Workflows

Orchestrate workflows that span multiple clouds using a workflow engine. Use Apache Airflow, Argo Workflows, or a custom state machine in a message queue.

Define a workflow that ingests data from AWS S3, processes it on GCP Compute Engine, and stores results in Azure Blob Storage.

In Airflow, define a DAG that runs daily:

from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime, timedelta

dag = DAG(
    'data-ingestion',
    default_args={
        'owner': 'data-team',
        'retries': 3,
        'retry_delay': timedelta(minutes=5),
    },
    description='Ingest data from AWS S3 to Azure Blob Storage via GCP',
    schedule_interval=timedelta(days=1),
    start_date=datetime(2025, 4, 1),
)

ingest_from_aws = BashOperator(
    task_id='ingest_from_aws_s3',
    bash_command="""
        aws s3 cp s3://data-ingest-bucket/daily/ /tmp/data/ --recursive
    """,
    dag=dag,
)

process_on_gcp = BashOperator(
    task_id='process_on_gcp',
    bash_command="""
        gcloud compute ssh gcp-worker-1 --command='python3 process.py /tmp/data'
    """,
    dag=dag,
)

upload_to_azure = BashOperator(
    task_id='upload_to_azure_blob',
    bash_command="""
        az storage blob upload-batch \
            --source /tmp/data/output \
            --destination azure-container \
            --account-name my-azure-storage
    """,
    dag=dag,
)

ingest_from_aws >> process_on_gcp >> upload_to_azure

A failure: the process_on_gcp task times out. The issue is that gcloud compute ssh connects via the default ssh configuration, but the GCP instance requires a custom SSH key and port.

Fix: configure gcloud with explicit SSH options:

gcloud config set compute/ssh_key_file ~/.ssh/gcp-ssh-key
gcloud config set compute/ssh_port 2222

A silent failure: the workflow runs, but the upload_to_azure task fails to authenticate to Azure. The az CLI command returns ERROR: The credentials file is not found. The issue is that the az CLI expects a ~/.azure/credentials file, but the service is running with azure as the user.

Fix: set AZURE_CONFIG_DIR environment variable:

export AZURE_CONFIG_DIR=/home/azure/.azure

Manage Data Movement and Replication

Manage data movement between clouds using standardized tools. Use aws s3 sync, gsutil, and azcopy for bulk transfers.

Move data from AWS S3 to GCP Cloud Storage:

aws s3 sync s3://my-data-bucket/ /tmp/data/ --region us-east-1
gsutil -m rsync -r /tmp/data/ gs://my-gcp-data-bucket/

Use rsync with --checksum for incremental transfers:

rsync -avz --checksum --delete \
  /data/ \
  user@gcp-worker:/data/

For real-time replication, use a message queue like Kafka or RabbitMQ.

A common failure: data is written to AWS S3 but not visible in GCP Cloud Storage after gsutil rsync. The issue is that gsutil uses a default chunk_size of 16MB, but the files are small, so gsutil uploads them as individual objects, not as a single file.

Fix: set chunk_size and parallel_composite_upload_threshold:

gsutil -m rsync -r \
  -c -z txt,log,json,xml \
  -o "chunk_size=8388608" \
  -o "parallel_composite_upload_threshold=10485760" \
  /tmp/data/ gs://my-gcp-data-bucket/

Handle Network and Latency Across Clouds

Design network architecture to minimize latency and maximize availability across clouds.

Use a global load balancer, such as Cloudflare or AWS Global Accelerator, to route requests to the nearest instance.

Configure DNS with weighted round-robin and health checks. Use dig to verify:

dig +short A www.company.com

The response should return multiple IP addresses, each corresponding to a cloud region.

Use tcpdump to monitor network traffic:

tcpdump -i any -n -s 0 -w /tmp/network.pcap port 80

A failure: a request to api.company.com takes 200ms, but the service logs show 10ms processing time. The issue is that the request is routed to AWS, but the database is in GCP, and the network latency is not accounted for.

Fix: use ping and traceroute to measure latency:

ping -c 10 api.company.com
traceroute -m 30 api.company.com

The traceroute output shows that the request crosses the internet between AWS and GCP with a 150ms RTT. The solution is to place a database replica in AWS and use a cross-cloud replication strategy.

Enforce Consistency with Policy as Code

Enforce configuration and operational consistency using policy as code. Use Open Policy Agent (OPA) to define policies that validate infrastructure and runtime behavior.

Define a policy that ensures every service runs on at least two instances across clouds:

package cloud.policy

default allow = false

allow {
  count(instances) >= 2
  all_instances_on_different_clouds
}

instances = [i | i := input.instances[_]]

all_instances_on_different_clouds {
  cloud_set = {i.cloud | i := input.instances[_]}
  count(cloud_set) >= 2
}

# Enforce that every service has a health check
health_check_required {
  input.service.health_check.path == "/health"
  input.service.health_check.interval_seconds == 30
}

Apply the policy using opa CLI:

opa eval -f pretty -i input.json -r policy.rego 'allow'

A failure: a service is deployed but the policy returns false with a detailed report:

{
  "result": false,
  "explanation": {
    "reason": "Less than 2 instances",
    "instances": [
      { "name": "user-profile-aws-1", "cloud": "aws" }
    ],
    "cloud_set": [ "aws" ]
  }
}

The service was deployed with only one instance in AWS, and the policy did not catch the missing GCP instance.

Use the policy in a CI pipeline:

- name: Validate Cloud Configuration
  run: |
    opa eval -f pretty -i config.json -r policy.rego 'allow'
    if [ $? -ne 0 ]; then
      echo "Policy validation failed"
      exit 1
    fi

This ensures that every deployment is validated against a shared set of cloud governance rules.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.