API Rate Limiting Patterns

Protect backend services from abuse and overload with rate limiting. Covers token bucket, sliding window, distributed rate limiting, client-specific limits, and the patterns that keep your API available under any traffic pattern.

API rate limiting is a fundamental mechanism for protecting backend services from overload, ensuring predictable performance, and enabling fair access across clients. It matters most when multiple consumers—web apps, mobile clients, third-party integrations, or internal microservices—access a shared API under variable load. This page details the core patterns for implementing rate limiting in APIs, focusing on the decisions engineers face at design and operational scale, with concrete configuration, error handling, and failure modes.

Per-Client Rate Limiting with Fixed Window

When a client—identified by API key, IP address, or OAuth access token—must be capped at a specific number of requests within a fixed time interval, the fixed window pattern is the simplest and most widely adopted.

The client sends requests with an Authorization: Bearer <token> header. The API checks a counter in a key-value store (e.g., Redis or Memcached) under a key like rate_limit:<client_id>:<window>, where <window> is the current time rounded to a fixed interval (e.g., 1 minute). Each request increments the counter.

A typical configuration in a service using Redis:

SET rate_limit:abc123:2025-04-05T12:30:00 1 EX 60

The EX 60 sets the key to expire after 60 seconds, aligning with a 1-minute window. The service checks the counter before processing a request:

func handleRequest(w http.ResponseWriter, r *http.Request) {
    clientID := extractClientID(r)
    window := time.Now().Truncate(time.Minute).Format("2006-01-02T15:04:00")

    key := fmt.Sprintf("rate_limit:%s:%s", clientID, window)
    count, err := client.Get(key).Int()
    if err != nil {
        http.Error(w, "Internal server error", http.StatusInternalServerError)
        return
    }

    if count >= 100 {
        w.Header().Set("Retry-After", "60")
        w.Header().Set("X-RateLimit-Limit", "100")
        w.Header().Set("X-RateLimit-Remaining", "0")
        w.Header().Set("X-RateLimit-Reset", time.Now().Add(60*time.Second).Format(time.RFC3339))
        w.WriteHeader(http.StatusTooManyRequests)
        json.NewEncoder(w).Encode(map[string]string{
            "error": "rate_limit_exceeded",
            "message": "Rate limit exceeded. Please retry after 60 seconds.",
            "retry_after": "60",
        })
        return
    }

    // Increment counter
    client.Incr(key).Result()
    w.Header().Set("X-RateLimit-Remaining", strconv.Itoa(100 - count - 1))
    w.Header().Set("X-RateLimit-Limit", "100")
    w.Header().Set("X-RateLimit-Reset", time.Now().Add(60*time.Second).Format(time.RFC3339))
}

This pattern fails silently when window boundaries are misaligned. For example, if a client hits 100 requests exactly at 12:30:00, the next request at 12:30:01 is counted in the new window, but the counter is not updated until the next tick. The window rolls every minute, but the counter is not updated atomically. A client can burst 101 requests in a single second just after a window boundary, exceeding the limit despite being within the window.

Sliding Window with Bucketing

For finer-grained control, the sliding window pattern avoids the burstiness of fixed windows by tracking individual request timestamps within a time window.

Each client’s request history is stored as a list of timestamps. On each request, the API:

  1. Fetches the list for the client.
  2. Filters entries older than the window (e.g., 60 seconds).
  3. Counts the remaining entries.
  4. Adds the current request timestamp.
  5. Stores the updated list.

Using Redis, a client’s history is stored as a sorted set (ZSET) with timestamps as members and scores:

ZADD rate_limit_history:abc123 1712345400 1712345400
ZADD rate_limit_history:abc123 1712345401 1712345401
ZADD rate_limit_history:abc123 1712345402 1712345402

The service checks the count of entries within the last 60 seconds:

def is_rate_limited(client_id, window=60):
    now = time.time()
    client_history = redis_client.zrangebyscore(f"rate_limit_history:{client_id}", now - window, now)
    count = len(client_history)
    if count >= 100:
        return True, {
            "error": "rate_limit_exceeded",
            "message": f"Rate limit exceeded: {count} requests in the last {window} seconds.",
            "retry_after": window - (now - float(client_history[-1])),
        }
    return False, {"count": count}

This pattern prevents burst spikes across window boundaries but introduces latency and storage cost. The ZADD operation is O(log N) per request, and the sorted set grows without bound unless cleaned. A common failure mode is missing the cleanup of old timestamps. Without periodic pruning, the set can grow to hundreds of thousands of entries per client, leading to slower ZRANGEBYSCORE calls and increased memory pressure.

Token Bucket with Dynamic Leaking

When clients need to handle bursty traffic and the API must allow temporary overloads, the token bucket pattern is ideal. It maintains a reservoir of tokens that refill at a steady rate.

Each client has a bucket with:

On each request, the service:

  1. Calculates time elapsed since the last refill.
  2. Computes new tokens added: tokens_added = (now - last_refill) * refill_rate.
  3. Updates tokens and last_refill.
  4. Deducts one token.
  5. If tokens < 1, the request is rate-limited.

The state is stored in a database table:

CREATE TABLE rate_limit_buckets (
    client_id VARCHAR(64) PRIMARY KEY,
    tokens FLOAT NOT NULL,
    last_refill TIMESTAMP NOT NULL,
    capacity INT NOT NULL,
    refill_rate FLOAT NOT NULL
);

A typical implementation:

-- Retrieve bucket state
SELECT client_id, tokens, last_refill, capacity, refill_rate
FROM rate_limit_buckets
WHERE client_id = 'abc123';

-- Update bucket: refill and consume
UPDATE rate_limit_buckets
SET tokens = tokens + (EXTRACT(EPOCH FROM NOW() - last_refill) * refill_rate),
    last_refill = NOW()
WHERE client_id = 'abc123';

-- Consume one token
UPDATE rate_limit_buckets
SET tokens = tokens - 1
WHERE client_id = 'abc123' AND tokens >= 1;

-- Check if token available
SELECT tokens >= 1 FROM rate_limit_buckets WHERE client_id = 'abc123';

This pattern handles bursts well—clients can issue 100 requests in one second if they have 100 tokens in their bucket. However, the refill rate must be carefully tuned. A refill rate of 100 tokens per minute means 100 requests per hour, but if the bucket starts empty, the first 100 requests are processed immediately, but the next 100 take 60 seconds to refill.

A common failure is missing the update of last_refill during refill, leading to under-refilling. If the service reads the bucket, computes new tokens, but fails to write last_refill, the next refill will start from an outdated timestamp, causing tokens to be added too slowly.

Hierarchical Rate Limiting with Tiered Plans

For multi-tenant systems, clients may have different access tiers: free, basic, premium. Hierarchical rate limiting applies multiple limits across tiers, with higher tiers overriding lower ones.

Each client is assigned a tier, and the API enforces a limit per tier. Requests are first checked against the free tier, then the basic tier, and so on.

A configuration might define:

The service checks the limits in order:

  1. Check free tier: use rate_limit:free:<client_id>:<window>.
  2. If exceeded, check basic tier: rate_limit:basic:<client_id>:<window>.
  3. If still exceeded, check premium tier.

Each check returns a X-RateLimit-* header and a Retry-After value. The service collects all headers and returns the most restrictive:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json

{
  "error": "rate_limit_exceeded",
  "message": "Tiered rate limiting exceeded.",
  "details": [
    {
      "tier": "free",
      "limit": 100,
      "remaining": 0,
      "reset": "2025-04-05T12:31:00Z",
      "retry_after": 60
    },
    {
      "tier": "basic",
      "limit": 500,
      "remaining": 450,
      "reset": "2025-04-05T12:31:00Z",
      "retry_after": 10
    }
  ]
}

The critical failure mode is tier overlap. If a client is in both basic and premium tiers, but the premium limit is checked first, the free tier might be satisfied but the basic tier not. The service must ensure the checks are evaluated in the correct order and that higher tiers override lower ones.

Leaky Bucket with Request Queueing

When a client exceeds its rate limit and requests are not rejected immediately but queued for processing, the leaky bucket with queueing pattern becomes essential.

The API maintains a queue of incoming requests for each client. The queue is processed by a worker that pulls requests at a fixed rate (the "leak rate"). Requests that arrive when the bucket is full are queued.

A typical queue is implemented using a message broker like RabbitMQ or Kafka.

Each request is published to a topic:

{
  "client_id": "abc123",
  "request_id": "req-456",
  "timestamp": "2025-04-05T12:30:00Z",
  "method": "GET",
  "path": "/api/v1/users"
}

A worker consumes from the queue and processes requests at a fixed rate (e.g., one request per second). The worker maintains a counter for each client and updates a shared state:

def worker():
    while True:
        message = queue.consume()
        client_id = message["client_id"]
        now = time.time()

        # Retrieve current state
        state = db.execute(
            "SELECT tokens, last_refill FROM rate_limit_state WHERE client_id = %s",
            (client_id,)
        ).fetchone()

        # Update state
        tokens = state["tokens"] + (now - state["last_refill"]) * leak_rate
        db.execute(
            "UPDATE rate_limit_state SET tokens = %s, last_refill = %s WHERE client_id = %s",
            (tokens, now, client_id)
        )

        # Process request
        if tokens >= 1:
            db.execute("INSERT INTO processed_requests (client_id, request_id) VALUES (%s, %s)", (client_id, message["request_id"]))
            db.execute("UPDATE rate_limit_state SET tokens = tokens - 1 WHERE client_id = %s", (client_id,))
        else:
            queue.enqueue(message, priority=1)  # Requeue for later processing

This pattern allows clients to absorb bursts and smooths out load. However, it introduces latency. A request that arrives during a burst may wait seconds before being processed. The failure mode is queue overflow: if the queue is not monitored, the backlog grows, leading to increased latency and potential memory exhaustion.

Header and Error Standardization

Rate limiting across services benefits from standardized response headers and error payloads.

The following headers are consistently used:

The error response body is structured to be machine-readable:

{
  "error": "rate_limit_exceeded",
  "message": "The client has exceeded its rate limit.",
  "details": {
    "limit": 100,
    "remaining": 15,
    "reset": "2025-04-05T12:31:00Z",
    "retry_after": 60,
    "window": "minute"
  },
  "links": {
    "documentation": "https://api.example.com/docs/rate-limiting"
  }
}

When a client receives a 429 Too Many Requests, it should:

  1. Parse Retry-After.
  2. Respect X-RateLimit-Reset.
  3. Use X-RateLimit-Remaining to estimate load.
  4. Retry with exponential backoff, using Retry-After as a base delay.

The most common failure is ignoring Retry-After in favor of a fixed delay. A client that assumes 60 seconds every time, but receives Retry-After: 120, may retry too early and miss the window.

Failure Modes and Tuning

Rate limiting is not static. The following failure modes appear in production:

Tuning involves:

Rate limiting is not a one-time setup but a continuous balancing act between performance, client experience, and system stability.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.