Production engineering guide for multi region active active covering patterns, implementation strategies, and operational best practices.
Multi Region Active Active (MRAA) is a cloud architecture pattern where multiple geographically distributed data centers operate simultaneously as primary, handling live traffic with no single point of failure. It ensures high availability, low latency, and automatic disaster recovery by distributing workloads across regions, each capable of serving all users. This pattern matters when an application must remain available through regional outages, meet stringent latency requirements, or scale beyond a single region’s capacity.
To deploy MRAA, you must first define a global service endpoint that routes traffic to the nearest active region. This is typically done using a DNS-based load balancing service that supports health checks and geolocation routing. The endpoint resolves to a set of region-specific IP addresses, each associated with a load balancer or API gateway in that region.
# Route 53 Record Set: api.example.com
{
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "us-east-1",
"Region": "us-east-1",
"GeoLocation": {
"ContinentCode": "NA",
"CountryCode": "US"
},
"ResourceRecords": [
"192.0.2.1"
],
"TTL": 60,
"HealthCheckId": "hc-1234567890"
}Each region runs a full instance of the application stack: compute (containers or virtual machines), stateful services (databases, caches), and shared state via a globally consistent data layer. The primary challenge is ensuring all regions can serve requests independently, with data consistency across regions.
Data consistency is achieved through a combination of replication, conflict resolution, and eventual consistency. The most common approach is to use a distributed database with multi-master replication, where writes to any region are propagated to all others.
For example, using PostgreSQL with logical replication and a conflict resolution layer:
-- Enable logical replication on each region's PostgreSQL instance
CREATE PUBLICATION mraa_publication FOR ALL TABLES;
-- Subscribe to all publications from other regions
CREATE SUBSCRIPTION mraa_subscription
CONNECTION 'host=us-east-1.example.com port=5432 dbname=app user=replica'
PUBLICATION mraa_publication
WITH (copy_data = true, create_slot = true);The replication stream ensures that changes in one region are applied in others. However, conflicts arise when two regions update the same record concurrently. A conflict resolution strategy must be implemented, typically in application code or via a middleware layer.
// Conflict resolution in Go for concurrent updates to a user profile
func ResolveConflict(user *UserProfile, sourceRegion string) {
if user.LastUpdated.Before(user.ReplicaLastUpdated) {
// Local region wins
user.SourceRegion = sourceRegion
} else if user.LastUpdated.After(user.ReplicaLastUpdated) {
// Remote region wins
user.SourceRegion = user.ReplicaSourceRegion
} else {
// Tie: use region name as tiebreaker
if sourceRegion < user.ReplicaSourceRegion {
user.SourceRegion = sourceRegion
} else {
user.SourceRegion = user.ReplicaSourceRegion
}
}
// Write resolved version back to database
db.Exec("UPDATE users SET source_region = $1, last_updated = $2 WHERE id = $3",
user.SourceRegion, user.LastUpdated, user.ID)
}The system must also handle replication lag and out-of-order delivery. A common failure mode is a region becoming temporarily unavailable during a write, leading to data divergence. To detect this, each region periodically reports its replication lag to a global monitoring service.
In MRAA, regional failures must be detected quickly and traffic rerouted with minimal impact. This involves health checks, region monitoring, and automated failover.
A region is considered healthy if its API gateway responds to a /health endpoint within 200ms and its database is reachable. A health check is executed every 10 seconds from a global monitoring service.
// Example health check response from a region
{
"status": "healthy",
"timestamp": "2023-10-05T12:00:00Z",
"latency": 150,
"database": {
"connected": true,
"replication_lag": 3500,
"last_checkpoint": "2023-10-05T11:59:57Z"
},
"services": [
{
"name": "auth-service",
"healthy": true,
"uptime": 99.8
},
{
"name": "user-service",
"healthy": false,
"error": "timeout",
"last_error": "2023-10-05T11:59:56Z"
}
]
}When a region’s health check fails three times consecutively, the global DNS service marks it as unhealthy and stops routing traffic to it. Traffic is then shifted to other regions using weighted round robin or geolocation-based routing.
# Example: Update Route 53 record to reduce weight for failed region
aws route53 update-resource-record-sets \
--hosted-zone-id Z1234567890 \
--change-batch '{
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "api.example.com",
"Type": "A",
"SetIdentifier": "us-west-2",
"Weight": 50,
"TTL": 60
}
}
]
}'If a region is completely down, the remaining regions must handle the additional load. This requires a capacity planning strategy that ensures each region can sustain 150% of its typical load.
In MRAA, shared resources such as session storage, message queues, and configuration stores must be accessible from all regions. A common approach is to use a globally accessible, replicated service.
For example, Redis used as a session store with Redis Cluster across regions:
# Redis Cluster setup with nodes in multiple regions
redis-cli --cluster create \
us-east-1.example.com:6379 \
us-west-2.example.com:6379 \
eu-central-1.example.com:6379 \
--cluster-replicas 1Each region connects to the Redis Cluster and uses it for session storage. When a user logs in to a region, their session is stored in Redis, and subsequent requests from any region can retrieve it.
Message queues are managed via a globally available broker such as Apache Kafka or AWS Kinesis. Producers in any region publish messages to a global topic, and consumers in all regions subscribe to it.
// Example Kafka message for user creation
{
"event_type": "user.created",
"timestamp": "2023-10-05T12:00:00Z",
"region": "us-west-2",
"user_id": "u-12345",
"email": "[email protected]",
"metadata": {
"source": "web",
"ip": "192.0.2.100"
}
}Consumers in each region process the message and update their local state. For example, the user-service in us-east-1 processes the event and creates a new user record in its local PostgreSQL instance.
Configuration must be consistent across regions. A centralized configuration service, such as a shared database or a distributed key-value store, is used to store and version configuration.
# Configuration stored in a shared database, accessed via HTTP
GET /config/v1/user-service?region=us-east-1
{
"version": "2.3.1",
"settings": {
"cache_ttl": 300,
"max_concurrent_requests": 256,
"feature_flags": {
"new_payment_flow": true,
"profile_images": false
}
},
"last_updated": "2023-10-05T11:55:00Z"
}Deployments are coordinated using a global CI/CD pipeline. A new version of the service is built and deployed to all regions in sequence, with a canary phase.
# GitHub Actions workflow for MRAA deployment
name: Deploy MRAA
on:
push:
branches: [ main ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: npm run build
- run: |
docker build -t app:v1.2.0 .
docker push app:v1.2.0
- run: |
# Deploy to us-east-1
kubectl rollout restart deployment/user-service -n us-east-1
sleep 60
# Deploy to us-west-2
kubectl rollout restart deployment/user-service -n us-west-2
# Deploy to eu-central-1
kubectl rollout restart deployment/user-service -n eu-central-1
- run: |
# Validate all regions
curl -f http://us-east-1.example.com/health
curl -f http://us-west-2.example.com/health
curl -f http://eu-central-1.example.com/healthA failure during deployment—such as a database migration that fails in one region—requires a rollback strategy. Rollback is triggered if the health check fails for more than 5 minutes after deployment.
A silent but common failure is replication lag exceeding the acceptable threshold. A region may serve stale data for minutes, leading to inconsistent user experiences.
# Example: Check replication lag across regions
for region in us-east-1 us-west-2 eu-central-1; do
echo "Region: $region"
psql -h $region.example.com -c "SELECT pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn(),
pg_wal_lsn_diff(pg_last_wal_replay_lsn(), pg_last_wal_receive_lsn()) as lag"
doneAnother failure mode is divergent configurations. A deployment script may update the configuration store, but some regions fail to pick up the change due to cache or network issues.
A third failure is misconfigured health checks. A health check that only probes the API gateway but not the database may mark a region as healthy even when the database is down.
A subtle but critical issue is the order of operations during a region failure. When a region becomes unavailable, clients may continue writing to it, but those writes are not immediately visible to other regions. This leads to a "write-ahead" state where data is written locally but not globally.
To handle this, a region must be marked as "read-only" after becoming unhealthy, and a background process must batch and forward writes to other regions.
Observability in MRAA requires a unified view across regions. Key metrics include:
Logs from all regions are aggregated into a central logging system such as Fluentd or Loki.
// Example log entry from a region
{
"timestamp": "2023-10-05T12:00:01Z",
"level": "error",
"service": "user-service",
"region": "us-west-2",
"message": "Failed to create user: duplicate email",
"trace_id": "a1b2c3d4e5f6",
"context": {
"user": {
"email": "[email protected]",
"ip": "192.0.2.200"
},
"replication_lag_ms": 4200
}
}Alerts are triggered when a region’s health check fails, replication lag exceeds 5 seconds, or error rate exceeds 2%. A global dashboard displays the state of each region, including current traffic load, active services, and ongoing deployments.
Multi Region Active Active is not a simple deployment strategy—it is a system of interdependent components that must be orchestrated across time and space. It demands careful attention to data consistency, failure detection, configuration synchronization, and observability. The most common failures are not abrupt outages but subtle divergences: stale data, misconfigured health checks, divergent deployments, and delayed replication. Mastering MRAA means building a resilient, globally distributed system where every region is a first-class citizen.
This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy