System Design Cheatsheet

System design is the art of turning requirements into scalable, reliable, and maintainable infrastructure. This cheatsheet guides engineers through the essential decisions, tools, and patterns for designing systems that work under real load—when the database slows, the network flaps, and the service fails in production. It is not a tutorial, but a field manual: use it during design sessions, architecture reviews, and on-call rotations.

Planning the System

Define Scope and Requirements

Start with a clear statement of what the system must do. Write the requirements in user-centric, measurable terms.

• "Users can upload and view photos with captions."
• "The system supports 10,000 concurrent users with sub-second latency."
• "All photos must be available within 5 seconds of upload."

Use the S.M.A.R.T. criteria implicitly: Specific, Measurable, Achievable, Relevant, Time-bound. Do not assume. Ask:

• What is the expected request rate?
• What is the data size per request?
• How much data is stored, and how fast does it grow?
• What are the availability and durability requirements?

Identify Key Components

Break the system into distinct parts:

• Frontend: Web, mobile, or desktop client.
• API Layer: REST or GraphQL endpoints.
• Application Logic: Business services.
• Data Layer: Databases, caches, message queues.
• Infrastructure: Load balancers, CDNs, containers, VMs.
• Monitoring and Observability: Logging, metrics, tracing.

Map components to services using Service Contracts:

• Define input and output formats using JSON schemas or OpenAPI 3.0.
• Use HTTP status codes consistently:
• 200 OK for success
• 201 Created after insert
• 400 Bad Request with error details in {"error": "invalid_field", "details": "name must be non-empty"}
• 429 Too Many Requests with Retry-After header
• 503 Service Unavailable with X-Backoff header

Designing for Scale

Choose a Deployment Strategy

Select a deployment pattern based on traffic and risk tolerance.

• Blue-Green Deployment:
• Two identical environments: blue (live), green (new).
• Deploy new version to green.
• Test green.
• Switch traffic with DNS or load balancer.
• Promote blue to green after success.
• Use health checks on /health endpoint.
• Failover: if green fails, switch back to blue.

• Canary Release:
• Deploy new version to 10% of users.
• Monitor metrics: latency, error rate, request throughput.
• Gradually increase traffic: 10% → 30% → 70% → 100%.
• Roll back if error rate exceeds 5% or latency > 200ms.

Use feature flags to control rollout:

• Enable /api/v1/users/{id} only for users with feature_flag: "new-user-api".
• Store flags in a database or in-memory cache.
• Apply flags at request time:

  {
    "user_id": 123,
    "name": "Alice",
    "features": {
      "new-user-api": true,
      "dark-mode": false
    }
  }

Choose a Database Strategy

Select a Database Type

• Use relational databases (PostgreSQL, MySQL) for:
• ACID transactions
• Complex queries
• Schema evolution
• Foreign key constraints
• Example: CREATE TABLE users (id SERIAL PRIMARY KEY, email VARCHAR(255) UNIQUE, created_at TIMESTAMPTZ);

• Use document stores (MongoDB, DynamoDB) for:
• Flexible schema
• High write throughput
• Nested data structures
• Example:

    {
      "user_id": 123,
      "profile": {
        "name": "Alice",
        "preferences": {
          "theme": "dark",
          "notifications": ["email", "push"]
        }
      },
      "last_login": "2024-03-15T10:00:00Z"
    }

Scale the Database

• Vertical Scaling: Increase VM size (CPU, RAM, disk).
• Simple, low cost.
• Limited by single machine.
• Use ALTER TABLE users ADD COLUMN bio TEXT; to evolve schema.

• Horizontal Scaling:
• Sharding: Distribute data across multiple databases.
• Shard by user_id:
• user_id % 16 = 0 → DB0
• user_id % 16 = 1 → DB1
• ...
• Use shard key in queries: SELECT * FROM users WHERE user_id = 123;
• Handle cross-shard joins with application-level joins or distributed query engines.

• Replication:
• One primary (write) node.
• Multiple replicas (read) nodes.
• Use read replicas for read-heavy workloads:
• SELECT * FROM users WHERE id = 123; → read from replica
• INSERT INTO users (email) VALUES ('[email protected]'); → write to primary
• Use read/write splitting:
• SELECT * FROM users; → read from replica
• UPDATE users SET name = 'Bob' WHERE id = 123; → write to primary

• Failover:
• Monitor primary with heartbeat (ping every 1s).
• If primary fails, promote first replica to primary.
• Update DNS or load balancer to point to new primary.
• Use read-only mode during failover:
• UPDATE pg_settings SET value = 'on' WHERE name = 'read_only';

Managing Data Flow

Introduce Message Queues

Use message queues (Kafka, RabbitMQ, Amazon SQS) to decouple components.

• Producer sends events to queue:

  {
    "event_type": "user_registered",
    "user_id": 123,
    "timestamp": "2024-03-15T10:00:00Z"
  }

• Consumer processes messages from queue:
• Pull messages from /api/v1/queue/consume
• Acknowledge after processing: {"ack_id": "abc123", "status": "processed"}
• Handle at-least-once delivery:
• Message processed → send ACK
• If ACK lost → message retried
• Idempotent processing: duplicate processing doesn’t break state

• Use dead-letter queues for failed messages:
• After 3 retries, move message to errors/dead-letter queue
• Use X-Message-Id, X-Redelivered, X-Attempts headers

Use Caching Strategically

Cache Layers

• In-Memory Cache (Redis, Memcached):
• Store frequently accessed data: user profiles, product listings
• Use GET /api/v1/users/123 → cache key: user:123
• Set TTL: EXPIRE user:123 3600 (1 hour)
• Handle cache misses:
• Fetch from database
• Store in cache
• Return to client

• CDN (Content Delivery Network):
• Cache static assets: images, CSS, JS, JSON
• Use Cache-Control: public, max-age=3600
• Invalidate cache via PURGE /images/avatar.jpg
• Support edge functions: execute logic at CDN edge nodes (e.g., resize image on-demand)

Cache Invalidation Strategies

• Write-Through:
• Write to cache and database simultaneously
• On PUT /api/v1/users/123:
• Update user:123 in Redis
• Insert/Update in PostgreSQL
• Return success

• Write-Behind:
• Write to cache first, then asynchronously write to database
• Use batching: flush every 100 writes or every 5s
• Handle cache staleness:
• GET /api/v1/users/123 → return cached data
• If stale, fetch from DB, update cache, return

• Cache-Aside (Lazy Loading):
• Check cache first
• If miss, load from DB, store in cache
• Update cache on write
• Use TTL and TTL + refresh-on-read

Ensuring Reliability

Handle Failures Gracefully

Network and Service Failures

• Use timeouts and retries in service calls:
• GET /api/v1/users/123 with timeout 5s
• Retry 3 times with exponential backoff: 1s, 2s, 4s
• Use X-Request-ID for tracing across services
• Log:

    {
      "timestamp": "2024-03-15T10:00:00Z",
      "level": "error",
      "service": "user-api",
      "request_id": "abc123",
      "message": "Failed to fetch user from database",
      "error": {
        "type": "DatabaseError",
        "code": "500",
        "details": "Connection timeout after 5000ms"
      }
    }

Load Balancing

• Use round-robin, least connections, or IP hash load balancing.
• Configure health checks:
• GET /health → return {"status": "UP", "version": "1.2.0"}
• Mark node as unhealthy if 3 consecutive failures
• Remove from pool until healthy

Circuit Breaker Pattern

• Wrap a remote service call with a circuit breaker:
• Closed: requests pass through
• Open: requests fail immediately
• Half-Open: allow limited number of requests; if success rate > 70%, close circuit

• Use Hystrix-style configuration:

  circuit_breaker:
    failure_threshold: 5
    timeout: 1000
    retry_attempts: 3
    retry_delay: 200
    half_open_threshold: 2

• Example:
• GET /api/v1/orders/456 → fails 5 times → open circuit
• Subsequent requests return 503 Service Unavailable immediately
• After 2s, allow 10 requests; if 7 succeed → close circuit

Monitoring and Observability

Logging

• Use structured logging:
• {"timestamp": "2024-03-15T10:00:00Z", "level": "info", "service": "api", "event": "user_login", "user_id": 123, "ip": "192.168.1.1"}
• Use log aggregation (ELK stack, Loki, Fluentd):
• Ship logs via journalctl --follow --unit=api.service
• Filter logs: level=error AND service=payment AND timestamp>2024-03-15T09:00:00Z

Metrics

• Expose metrics via Prometheus or StatsD:
• http_requests_total{method="GET", route="/api/v1/users"} 1234
• http_request_duration_seconds{method="POST", status="201"} 0.123
• Use Gauges, Counters, Histograms, Summaries

• Alert on key metrics:
• rate(http_requests_total[5m]) > 100 → high request rate
• avg_over_time(http_request_duration_seconds{method="GET"}[1h]) > 1s → slow API

Tracing

• Use distributed tracing (OpenTelemetry, Jaeger):
• Each request gets a trace_id
• Each service adds a span with span_id, operation, start, end, tags
• Example trace:

    trace_id: abc123
      span_id: 1
        operation: "HTTP GET /api/v1/users/123"
        start: 2024-03-15T10:00:00.123Z
        end: 2024-03-15T10:00:00.345Z
        tags: { user_id: 123, client: "mobile" }
      span_id: 2
        operation: "Database query: SELECT * FROM users WHERE id = $1"
        start: 2024-03-15T10:00:00.200Z
        end: 2024-03-15T10:00:00.300Z
        tags: { query: "SELECT * FROM users WHERE id = 123" }

• Use trace context in HTTP headers:
• traceparent: 00-abc123-123456-01
• tracestate: user_id=123,session_id=xyz

Final Checks

• Review for consistency:
• All services use same trace_id, request_id, user_id
• All error responses have error object with code, message, details
• All APIs use snake_case for JSON fields

• Validate deployment:
• Use curl -v http://api.example.com/health
• Check:
• HTTP/1.1 200 OK
• Content-Type: application/json
• Body: {"status": "UP", "version": "1.2.0", "uptime": 3600}

• Document the design:
• Create a system diagram using Mermaid, PlantUML, or hand-drawn
• Label services, databases, queues, caches, load balancers
• Include data flow arrows with request and event labels
• Add legend:
• API → REST/GraphQL
• DB → PostgreSQL
• Queue → Kafka
• Cache → Redis
• LB → nginx

This is the system design cheatsheet. Use it to build systems that are not just functional, but resilient, observable, and ready for growth.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.