Implement rate limiting that protects your APIs from abuse without degrading legitimate traffic. Covers token bucket, sliding window, distributed rate limiting, client-specific quotas, and graceful degradation strategies.
Rate limiting and throttling prevent systems from being overwhelmed by client requests, especially in APIs and microservices. When a client floods a service with calls—whether from a mobile app, a background job, or a third-party integration—the system must balance load, resource usage, and response latency. These patterns ensure that services remain responsive under varying traffic, protect downstream dependencies, and provide predictable performance. They are essential in any system where clients can burst, batch, or continuously poll, and where resources such as database connections, CPU, or network bandwidth are finite.
Apply rate limiting per client, per user, or per IP address to control how many requests a client can make in a fixed time window. The most common approach uses a sliding window algorithm with a token bucket or fixed window counter.
A client receives a 200 OK response with headers indicating the current rate limit state:
HTTP/1.1 200 OK
Content-Type: application/json
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 97
X-RateLimit-Reset: 1711234567The client uses X-RateLimit-Limit to learn the maximum allowed requests per window. X-RateLimit-Remaining tracks available tokens. X-RateLimit-Reset is a Unix timestamp indicating when the window resets.
After 97 requests, the client receives a 429 Too Many Requests response:
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 60
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1711234567
{
"error": "Rate limit exceeded",
"message": "You have reached the maximum number of requests per minute",
"retry_after": 60
}The Retry-After header suggests when to retry, either in seconds (60) or as a timestamp (1711234567).
Use a configuration file to define limits per route, client, or user:
rate_limits:
default:
window: 60s
limit: 100
strategy: sliding_window
storage: redis
key_template: "rate_limit:{client_id}:{path}"
burst: 20
api/v1/users:
window: 10s
limit: 50
strategy: fixed_window
storage: redis
key_template: "rate_limit:{user_id}:{path}"
burst: 10Each route applies its own rules. The sliding_window strategy uses Redis to maintain a list of timestamps for each request, allowing precise windowing. The fixed_window strategy increments a counter for each window period (e.g., every 10 seconds) and resets the counter at the interval boundary.
For clients that vary in behavior—some bursty, others steady—use dynamic windowing. The system tracks request arrival times and adjusts the window size based on traffic patterns.
When a client sends requests at irregular intervals, the service calculates the effective window using a sliding window of requests. It maintains a list of timestamps in a Redis sorted set:
ZADD rate_limit:client_123:api/v1/users 1711234500.123 1
ZADD rate_limit:client_123:api/v1/users 1711234512.456 1
ZADD rate_limit:client_123:api/v1/users 1711234520.001 1A query retrieves the count of requests within the last 60 seconds:
ZRANGEBYSCORE rate_limit:client_123:api/v1/users -inf 1711234560.0The response includes the count and the next available window. If the count exceeds the limit, the service returns 429 with X-RateLimit-Remaining: 0 and Retry-After: 30.
Clients sending 100 requests in 3 seconds must be allowed to burst beyond the base limit. The burst configuration enables a "token bucket" model where clients accumulate additional tokens beyond their base limit.
A client with limit: 50, burst: 20, and window: 10s can send 70 requests in a single burst, consuming 20 extra tokens. The system tracks two values:
tokens: current number of available tokenslast_updated: timestamp of the last updateOn each request, the system updates the token count:
def update_token_bucket(client_id, path, rate_limit_config):
# Get current state
tokens = redis.hget(f"bucket:{client_id}:{path}", "tokens")
last_updated = redis.hget(f"bucket:{client_id}:{path}", "last_updated")
# Calculate elapsed time
now = time.time()
elapsed = now - float(last_updated or now)
# Add new tokens
new_tokens = elapsed * rate_limit_config["rate"]
tokens += new_tokens
# Cap tokens
tokens = min(tokens, rate_limit_config["limit"] + rate_limit_config["burst"])
# Use one token
if tokens >= 1:
tokens -= 1
redis.hset(f"bucket:{client_id}:{path}", "tokens", tokens)
redis.hset(f"bucket:{client_id}:{path}", "last_updated", now)
return True, tokens
else:
return False, tokensThe client receives 429 with Retry-After set to the time needed to refill the bucket.
When multiple services share a common data layer—such as a database or a message queue—throttle at the system level. This prevents any single service from overwhelming the shared resource.
A central throttler service monitors request counts, response times, and resource utilization. It adjusts the rate limits on all services dynamically.
A GET /throttling/status endpoint returns the current state:
{
"services": {
"user-service": {
"rate_limit": 100,
"current_requests": 94,
"total_capacity": 200,
"throttled": true,
"reason": "Database load above 80%"
},
"order-service": {
"rate_limit": 75,
"current_requests": 75,
"total_capacity": 150,
"throttled": true,
"reason": "CPU utilization at 95%"
}
},
"global": {
"database_load": 0.85,
"message_queue_depth": 1200,
"cpu_utilization": 0.92
}
}When the database load exceeds 80%, the throttler sends a PATCH /throttling request to all services:
PATCH /throttling
Content-Type: application/json
{
"user-service": {
"rate_limit": 80,
"reason": "Database load increased"
},
"order-service": {
"rate_limit": 60,
"reason": "High CPU and queue depth"
}
}Each service updates its rate limit configuration and applies the new limits immediately.
In a Go microservice, a middleware function enforces rate limits before handling each request:
func RateLimitMiddleware(handler http.Handler, config RateLimitConfig) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
clientID := getClientID(r)
path := r.URL.Path
key := fmt.Sprintf("rate_limit:%s:%s", clientID, path)
client := redis.NewClient(&redis.Options{
Addr: "redis:6379",
})
// Get current count and window
val, err := client.Get(key).Result()
if err != nil || val == "" {
val = "0"
}
count, _ := strconv.Atoi(val)
window := config.Window
// Check if limit exceeded
if count >= config.Limit {
w.Header().Set("X-RateLimit-Limit", strconv.Itoa(config.Limit))
w.Header().Set("X-RateLimit-Remaining", "0")
w.Header().Set("Retry-After", strconv.Itoa(int(window.Seconds())))
w.WriteHeader(http.StatusTooManyRequests)
w.Write([]byte(`{"error":"Rate limit exceeded","message":"Request limit reached"}`))
return
}
// Increment counter
client.Incr(key)
client.Expire(key, window)
// Set headers
w.Header().Set("X-RateLimit-Limit", strconv.Itoa(config.Limit))
w.Header().Set("X-RateLimit-Remaining", strconv.Itoa(count+1))
w.Header().Set("X-RateLimit-Reset", strconv.FormatInt(time.Now().Add(window).Unix(), 10))
// Call next handler
handler.ServeHTTP(w, r)
})
}The middleware is applied globally:
mux := http.NewServeMux()
mux.Handle("/api/v1/users/", RateLimitMiddleware(userHandler, userConfig))
mux.Handle("/api/v1/orders/", RateLimitMiddleware(orderHandler, orderConfig))For systems with multiple request types—some time-critical, others batch-oriented—use prioritized throttling. High-priority requests are processed first and have their own rate limits.
A request includes a X-Priority header:
POST /api/v1/events
X-Priority: high
Content-Type: application/json
{
"event_type": "user_login",
"user_id": 12345,
"timestamp": "2024-03-20T10:00:00Z"
}The service maintains separate rate limit counters for each priority level:
high: limit 200 per minute, burst 30medium: limit 100 per minute, burst 15low: limit 50 per minute, burst 10Requests are processed in priority order. When a high-priority request arrives, the service checks if the high-priority bucket is full. If so, it enforces 429 with X-Priority-Status: throttled.
When a high-priority request is blocked by a long-running low-priority job, the system can apply priority boosting. The service maintains a priority queue of active requests and uses a worker pool.
Each worker polls a shared Redis list:
BRPOPLPUSH pending:high priority:high 30
BRPOPLPUSH pending:medium priority:medium 30
BRPOPLPUSH pending:low priority:low 30When a worker completes a high-priority task, it checks if the next high-priority request is ready:
LLEN priority:highIf the list is empty, the worker can idle, but if it’s non-empty, it processes the next request immediately.
A service uses a configuration file with per-priority settings:
rate_limits:
priority:
high:
window: 60s
limit: 200
burst: 30
storage: redis
key_template: "rate_limit:{client_id}:high:{path}"
medium:
window: 60s
limit: 100
burst: 15
storage: redis
key_template: "rate_limit:{client_id}:medium:{path}"
low:
window: 60s
limit: 50
burst: 10
storage: redis
key_template: "rate_limit:{client_id}:low:{path}"Clients use X-Priority to indicate their intent, and the system uses X-RateLimit-Remaining and Retry-After to guide them.
Throttling silently fails when the storage backend becomes unavailable. Redis or a database may be down, causing rate limit updates to be lost.
When a service cannot update the rate limit counter, it logs an error and falls back to in-memory tracking:
{
"error": "Rate limit update failed",
"service": "user-service",
"client_id": "client_123",
"path": "/api/v1/users",
"timestamp": "2024-03-20T10:05:30Z",
"error_code": "redis_unavailable",
"retry_after": 30
}If the storage fails consistently, the system increases the fallback window and sends alerts.
A common failure is misinterpreting window and reset. A client assuming a fixed window may retry too early if the system uses a sliding window. The X-RateLimit-Reset header must be accurate—its value is a Unix timestamp, not a duration.
Another sharp edge is the Retry-After header. If Retry-After: 60, the client expects to wait 60 seconds before retrying. But if the server resets the window at a different time, the client may need to adjust its X-RateLimit-Remaining value.
Finally, when multiple throttling layers exist—client, service, and global—clients must be aware of cumulative limits. A client sending 100 requests per minute may be throttled at the service level (limit 50) and also at the global level (limit 80). The client must track both limits and respond to 429 from either layer.
X-RateLimit-* headers to communicate limits and status.429 Too Many Requests with Retry-After to guide clients.Redis or a database for scalable, distributed rate limiting.These patterns ensure that systems remain stable, predictable, and responsive under load, and that clients can adapt to changing conditions without explicit configuration.
This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy