System Design Interview Patterns for Senior Engineers

This page describes patterns and anti-patterns that appear in system design interviews for senior engineering roles. These are the specific architectural decisions, trade-offs, and failure modes that distinguish candidates who can build systems from those who can only describe them. The patterns here are about how to approach problems when designing systems under constraints.

When to Apply These Patterns

These patterns help structure thinking when an interviewer asks you to design a system with given constraints. They're not about choosing specific frameworks or platforms, but about how to decompose problems, make trade-offs, and reason about scalability, reliability, and maintainability.

Designing for Scale and Availability

Distributing Workloads Across Multiple Instances

When designing systems that must handle high load, you must distribute work across multiple instances. The simplest approach is horizontal scaling with stateless services.

Example: A web application that serves requests using a load balancer with round-robin distribution.

# Typical HTTP server configuration
nginx.conf:
upstream backend {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

Failure mode: If an instance fails, the load balancer continues routing to healthy instances, but requests to that failed instance are lost until it recovers.

Handling Session State

Stateful applications must manage session persistence carefully. The simplest approach is to store session data in a shared database or cache.

Example: Using Redis for session storage with TTL expiration.

# Redis command to set a session with TTL
redis-cli SETEX session_12345 3600 "user_id:12345,role:admin"

Failure mode: If Redis becomes unavailable, new sessions cannot be created and existing sessions may time out.

Implementing Caching Strategies

Caching reduces load on primary systems. Two main patterns are cache-aside and write-through.

Example: Cache-aside pattern in a service that fetches user data.

def get_user(user_id):
    # Try cache first
    cached = redis.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)
    
    # Fetch from database
    user = db.query("SELECT * FROM users WHERE id = ?", user_id)
    # Cache for 5 minutes
    redis.setex(f"user:{user_id}", 300, json.dumps(user))
    return user

Failure mode: Cache invalidation delays mean stale data is returned until cache expires.

Designing for Reliability and Fault Tolerance

Implementing Circuit Breakers

Circuit breakers prevent cascading failures when downstream services are slow or failing.

Example: Using a simple circuit breaker implementation with timeout and failure threshold.

class CircuitBreaker:
    def __init__(self, failure_threshold=5, timeout=60):
        self.failure_threshold = failure_threshold
        self.timeout = timeout
        self.failures = 0
        self.last_failure_time = None

    def call(self, func, *args, **kwargs):
        if self.is_open():
            raise Exception("Circuit is open")
        try:
            result = func(*args, **kwargs)
            self.success()
            return result
        except:
            self.failure()
            raise

Failure mode: Circuit breaker may stay open longer than necessary if it's not configured with appropriate timeout values.

Handling Failures Gracefully

When a system component fails, graceful degradation is preferred over total failure.

Example: Service that continues to operate when external API becomes unavailable.

def get_recommendations(user_id):
    try:
        # Call external recommendation service
        response = requests.get("http://recommendation-service/api/recommend", timeout=2)
        return response.json()
    except requests.Timeout:
        # Fall back to cached or basic recommendations
        return {"items": ["item_1", "item_2"]}

Failure mode: If fallback logic is not implemented, all requests fail when the external service is down.

Managing Data Consistency

Handling Distributed Transactions

In distributed systems, strict ACID transactions are often replaced with eventual consistency patterns.

Example: Two-phase commit approach using a coordinator.

-- Phase 1: Prepare
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
UPDATE accounts SET balance = balance + 100 WHERE id = 2;
-- Write prepare record to log table
INSERT INTO transaction_log (id, status) VALUES (12345, 'prepared');

-- Phase 2: Commit
UPDATE transaction_log SET status = 'committed' WHERE id = 12345;

Failure mode: If the coordinator fails during commit phase, transactions may be left in inconsistent state.

Implementing Eventual Consistency

Eventual consistency is common in distributed systems. Events are published when changes occur.

Example: Using message queues to propagate changes.

# Producer publishes event
echo '{"event_type":"user_created","user_id":12345,"timestamp":1609459200}' | \
  rabbitmqctl publish exchange_name routing_key

# Consumer processes event
rabbitmqctl consume queue_name 'process_user_event'

Failure mode: If consumer fails to process an event, the system may not converge to consistent state.

Designing for Performance and Latency

Optimizing Query Patterns

Database queries must be designed with performance in mind. Indexes are critical for large datasets.

Example: Proper indexing for frequent queries.

-- Table with appropriate indexes
CREATE TABLE orders (
    id BIGINT PRIMARY KEY,
    user_id BIGINT,
    created_at TIMESTAMP,
    status VARCHAR(20)
);

CREATE INDEX idx_orders_user_created ON orders(user_id, created_at);
CREATE INDEX idx_orders_status ON orders(status);

Failure mode: Missing indexes cause full table scans and slow query performance.

Using Batch Processing

Batch processing reduces system load by grouping operations.

Example: Processing user updates in batches.

def batch_process_users(updates):
    # Group by user ID
    grouped = {}
    for update in updates:
        grouped.setdefault(update['user_id'], []).append(update)
    
    # Process each user's updates
    for user_id, user_updates in grouped.items():
        process_user_batch(user_id, user_updates)

Failure mode: Large batch sizes cause memory pressure and long processing times.

Planning for Data Growth

Implementing Sharding Strategies

Sharding distributes data across multiple database instances or partitions.

Example: Hash-based sharding of user data.

-- Shard key is user_id mod 4
SELECT * FROM users WHERE user_id % 4 = 0;

Failure mode: If sharding key is not chosen carefully, some shards may become hotspots with uneven load.

Handling Data Migration

Data migration requires careful planning to avoid downtime or data inconsistency.

Example: Rolling migration approach.

# Two-phase deployment strategy
# Phase 1: New code with read from both sources
# Phase 2: Redirect writes to new system
# Phase 3: Cleanup old system

Failure mode: If rollback plan is not in place, a failed migration can leave data in inconsistent state.

Designing for Operational Simplicity

Implementing Health Checks

Systems must expose health status to monitoring systems.

Example: HTTP endpoint for health checks.

# Health check endpoint
curl -f http://localhost:8080/healthz

Failure mode: Health checks that only test the application layer but ignore database connections may report false positives.

Implementing Graceful Shutdown

Services should allow graceful shutdown to avoid interrupting active requests.

Example: Signal handling for graceful shutdown.

import signal
import sys

def signal_handler(sig, frame):
    print('Shutting down gracefully...')
    # Stop accepting new requests
    # Finish processing current requests
    sys.exit(0)

signal.signal(signal.SIGTERM, signal_handler)

Failure mode: If a service doesn't handle SIGTERM, it may be killed abruptly and lose in-flight work.

Handling Concurrency and Race Conditions

Implementing Locking Mechanisms

When concurrent access to shared resources is necessary, locking mechanisms prevent race conditions.

Example: Using Redis locks for distributed coordination.

# Acquire lock
redis-cli SET resource_lock "locked" NX EX 10

# Release lock
redis-cli EVAL "if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) else return 0 end" 1 resource_lock "locked"

Failure mode: If lock acquisition fails, concurrent operations may proceed without coordination.

Managing Concurrent Writes

When multiple clients write to the same data, conflict resolution is needed.

Example: Using optimistic concurrency control with version numbers.

UPDATE products SET 
    name = 'new_name', 
    version = version + 1 
WHERE id = 123 AND version = 5;

Failure mode: If version check fails, updates are rejected and must be retried by the client.

Designing for Observability

Implementing Metrics Collection

Systems should expose metrics for monitoring.

Example: Exposing Prometheus metrics.

# Example with Python prometheus_client
from prometheus_client import Counter

request_count = Counter('http_requests_total', 'Total HTTP Requests')

def handle_request():
    request_count.inc()
    # Process request

Failure mode: If metrics are not collected or exported properly, system performance issues may go unnoticed.

Logging and Tracing

Proper logging and tracing help debug complex distributed systems.

Example: Structured logging with trace identifiers.

import logging

logger = logging.getLogger(__name__)

def process_order(order_id):
    logger.info("Processing order", extra={"order_id": order_id, "trace_id": "abc123"})

Failure mode: If logs don't include sufficient context for correlation, debugging becomes difficult.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.