Design caching strategies that reduce latency, lower database load, and scale horizontally. Covers cache-aside, read-through, write-behind, cache invalidation, distributed cache topologies, and the patterns that make caching reliable in production.
Distributed caching is essential when a backend system must serve data quickly across multiple services, nodes, or regions. It matters when latency from a database, file system, or external API dominates response times, and when multiple services access the same data frequently. This page describes the patterns for designing and operating distributed caches—how to structure data, manage consistency, handle failures, and scale efficiently across clusters.
The most common and fundamental pattern: applications first check the cache before accessing the underlying data store.
When a service needs data, it issues a GET request to the cache. If the key exists, it returns the value immediately. If not, it fetches the data from the database, then stores it in the cache before returning it to the caller.
GET /api/users/123
Host: cache.example.com// Response: Cache miss
HTTP/1.1 200 OK
Content-Type: application/json
X-Cache: MISS
{
"id": 123,
"name": "Alice",
"email": "[email protected]"
}After the database fetch, the service calls:
PUT /api/users/123
Host: cache.example.com
Content-Type: application/json
{
"id": 123,
"name": "Alice",
"email": "[email protected]"
}The cache is populated with a TTL (time-to-live) of 300 seconds.
The cache key is typically derived from the request path and query parameters. For a list of users with a filter:
GET /api/users?role=admin&status=activeThe key becomes users:role:admin:status:active.
A common failure: the cache is populated but the database is updated concurrently. A write operation may miss updating the cache, leading to stale data. To address this, the application must update the cache after a database write, either synchronously or asynchronously.
With write-through, every write to the database is immediately mirrored in the cache.
On POST /api/users, the service first writes to the database, then updates the cache:
POST /api/users
Host: db.example.com
Content-Type: application/json
{
"name": "Bob",
"email": "[email protected]"
}On success, it calls:
PUT /api/users
Host: cache.example.com
Content-Type: application/json
{
"id": 456,
"name": "Bob",
"email": "[email protected]"
}The cache is updated synchronously, so a PUT to the cache blocks the response until the cache accepts the data.
This pattern ensures cache consistency but adds latency to writes. If the cache is unavailable, the write fails and the entire operation is delayed.
The cache can be configured to accept writes with TTL and expire directives:
{
"key": "users:456",
"value": {
"id": 456,
"name": "Bob",
"email": "[email protected]"
},
"ttl": 600,
"expire": "2025-04-05T12:00:00Z"
}A subtle failure: if the cache is updated but the database write fails, the cache is inconsistent with the database. To recover, the system must either retry the database write or use a compensating update (e.g., via a message queue).
In write-behind, writes are recorded in the cache immediately and later flushed to the database asynchronously.
After a POST /api/users, the service updates the cache:
PUT /api/users
Host: cache.example.com
Content-Type: application/json
{
"id": 456,
"name": "Bob",
"email": "[email protected]"
}The response returns immediately to the caller. The cache stores a queue of pending writes, and a background process periodically flushes them to the database.
The queue may be implemented as a LIST or STREAM in the cache:
LPUSH cache:write_queue '{"table": "users", "id": 456, "data": { ... }}'
# Worker process: pop and apply
while true; do
read job in LPOP cache:write_queue
if [ -n "$job" ]; then
# Parse job
table=$(echo "$job" | jq -r '.table')
id=$(echo "$job" | jq -r '.id')
data=$(echo "$job" | jq -c '.data')
# Apply to database
psql -c "
INSERT INTO $table (id, data) VALUES ($id, $data)
ON CONFLICT (id) DO UPDATE SET data = EXCLUDED.data
"
fi
sleep 1
doneThis pattern reduces write latency and increases throughput but introduces eventual consistency. The cache may be updated while the database is offline, and a cache flush may be lost if the worker crashes.
A sharp edge: if the worker crashes between LPUSH and psql, the write is lost. To prevent this, use transactional operations in the cache (e.g., MULTI/EXEC in Redis) or log the write in a durable queue (e.g., Kafka or RabbitMQ).
The read-through pattern extends cache-aside by automatically loading missing data from the data store into the cache.
The service calls the cache for a key, but if the key is missing, the cache not only returns MISS, but also automatically calls a configured data loader to populate the cache.
The cache is configured with a loader function or endpoint:
{
"cache": {
"name": "user_cache",
"type": "redis",
"config": {
"host": "redis.example.com",
"port": 6379,
"read_through": {
"enabled": true,
"loader": {
"url": "http://loader-service.example.com/load",
"method": "GET",
"headers": {
"Accept": "application/json"
},
"timeout": 5000
}
}
}
}
}When a client requests GET /api/users/123, and the key users:123 is missing, the cache calls:
GET /load?table=users&id=123
Host: loader-service.example.com
Accept: application/jsonThe loader returns:
{
"id": 123,
"name": "Alice",
"email": "[email protected]"
}The cache then stores this data with a TTL of 300 seconds.
The loader can be a standalone service or a function triggered by a message. The response format is not fixed: the loader can return any JSON, and the cache applies it to the key.
Failure modes: if the loader is unreachable, the cache returns MISS but does not populate the cache. If the loader returns a malformed response, the cache may fail to parse the data. If the loader returns a 404, the cache may treat it as a MISS and retry with a fallback loader.
When data changes, the cache must be updated or invalidated to avoid serving stale data.
The simplest method is manual invalidation: after a write, the service removes or updates the relevant cache keys.
DELETE /api/users/123
Host: cache.example.comThis removes the key users:123 from the cache. A subsequent GET will miss and fetch the latest data.
For multiple related keys, use a batch invalidation:
DEL
users:123
users:124
users:125
users:role:adminThe cache accepts multiple keys in a single DEL command.
A common failure: the cache is invalidated, but the data is updated before the cache is reloaded. The new data may be stale by the time the cache is updated.
To address this, use cache stamping with versioned keys:
{
"users:123:12345": {
"id": 123,
"name": "Alice",
"email": "[email protected]"
},
"users:123:12346": {
"id": 123,
"name": "Alice",
"email": "[email protected]"
}
}Each write updates the users:123 key with a version number. The client reads from the versioned key and uses the last_modified timestamp to determine freshness.
A more advanced pattern: cache invalidation with pub/sub.
When a user is updated, the service publishes an event:
{
"event": "user.updated",
"data": {
"id": 123,
"version": 12346,
"timestamp": "2025-04-05T11:30:00Z"
}
}All caches subscribe to this event. On receipt, each cache invalidates all keys related to the user:
DEL
users:123
users:123:*
users:role:adminThis pattern scales to multiple services and regions, but it requires a reliable message broker (e.g., Kafka, RabbitMQ, or AWS SNS).
To reduce cold-start latency, caches are pre-populated with frequently accessed data during deployment or startup.
A service starts and calls a WARMUP endpoint:
POST /cache/warmup
Host: cache.example.com
Content-Type: application/json
[
"users:123",
"users:456",
"users:789",
"users:role:admin",
"products:latest"
]The cache returns a 202 Accepted response and begins loading the keys in the background.
The warmup process may be orchestrated via a job scheduler or a background worker.
A failure: if the warmup request includes a key that does not exist in the database, the cache returns MISS but does not populate the key. To fix this, the warmup process must fetch the data and insert it into the cache.
Another edge: the warmup process may be slow, and the service must be prepared to serve requests while the cache is warming. The cache may be partially warmed, and clients should be able to handle both HIT and MISS responses during the process.
In a distributed cache, multiple cache nodes serve the same data. Keys are distributed across nodes using consistent hashing.
The cache uses a KEY prefix to determine which node serves it:
users:123 → node-a
products:456 → node-b
orders:789 → node-cThe hash function maps keys to nodes:
def get_node(key: str) -> str:
hash_value = hash(key) % 100 # 100 slots
if hash_value < 30:
return "node-a"
elif hash_value < 60:
return "node-b"
else:
return "node-c"Each node maintains its own cache and responds to requests via a common interface.
When a key is missing, the client or a front-end proxy routes the request to the correct node.
Failure modes: if a node crashes, all keys it served become unavailable. To recover, the system must reassign keys to other nodes.
A sharp edge: if the hash function changes (e.g., increasing the number of nodes), keys must be migrated. The migration process must be incremental: new keys are added to the new nodes, and old keys are gradually moved.
To reduce migration cost, use virtual nodes (vnodes): each physical node manages multiple virtual nodes, and the hash space is divided into 100 vnodes. When a new node is added, it takes over a subset of vnodes, and the existing nodes adjust their load.
When cache memory is full, the system must evict old entries.
The most common policy is LRU (least recently used). The cache maintains a list of keys ordered by access time. When a new entry is added, the least recently used entry is removed.
Another policy is LFU (least frequently used): entries with fewer accesses are evicted first.
The cache may also support TTL-based eviction: entries are removed after a fixed time.
Configuration is done via a set of directives:
{
"cache": {
"eviction": "lru",
"max_entries": 100000,
"ttl": 300,
"lfu": {
"window": "1h",
"threshold": 10
}
}
}A subtle failure: if the cache is updated frequently but the access pattern is skewed (e.g., 90% of reads are for 10% of keys), LRU may not evict the least frequently used entries correctly.
To improve this, use adaptive eviction: the system monitors access patterns and switches between LRU and LFU based on load.
The success of a distributed cache is measured by its hit ratio: the fraction of requests that find data in the cache.
A hit ratio below 80% indicates that the cache is not effective. The system should monitor:
A typical metric is:
cache_hit_ratio: 0.87
cache_misses: 1423
cache_latency_avg: 12.3ms
cache_memory_usage: 85%A common failure: the cache is warm, but the hit ratio drops during peak hours. Investigation reveals that the TTL is too short, causing frequent misses. The solution is to increase the TTL or use dynamic TTLs based on data access patterns.
To detect this, the service logs every cache operation:
{
"timestamp": "2025-04-05T11:30:00Z",
"operation": "GET",
"key": "users:123",
"result": "HIT",
"latency_ms": 15,
"cache_node": "node-a"
}A cache miss is logged with the full stack trace and database query that followed.
Distributed caches support multiple consistency models.
GET, the cache synchronously reads from the database if the data is stale.The cache can be configured to use a consistency model by setting a consistency directive:
{
"cache": {
"consistency": "eventual",
"read_consistency": "strong",
"write_consistency": "eventual"
}
}A failure: a cache is configured for strong consistency, but a GET request is routed to a node that has not yet received a write. The cache returns stale data, but the client expects the latest version.
To fix this, the system must implement cache versioning and read-your-writes semantics.
When a cache node fails, the system must route requests to other nodes.
The cache cluster uses a discovery service or a configuration file to list available nodes:
{
"cache_nodes": [
"cache-a.example.com:6379",
"cache-b.example.com:6379",
"cache-c.example.com:6379"
],
"failover": {
"strategy": "round_robin",
"timeout": 5000,
"retry_attempts": 3
}
}If cache-a is unavailable, the system routes to cache-b and cache-c in a round-robin fashion.
A sharp edge: if a node is slow but not down, the system may failover unnecessarily. To avoid this, use health checks:
GET /health
Host: cache-a.example.comThe response includes:
{
"status": "healthy",
"uptime": 3600,
"load": 0.75,
"memory_usage": 80
}The system marks a node as unhealthy if it fails three consecutive health checks.
When a node fails, the system updates the configuration and notifies all clients. Clients may cache the configuration and refresh it periodically.
This page describes how to build, operate, and tune distributed caches. The patterns are not tied to a single platform but are applicable across systems. Each pattern can be composed with others to form a robust, scalable caching layer.
This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy