Implement caching that reduces latency from hundreds of milliseconds to single-digit milliseconds. Covers cache placement, invalidation strategies, cache-aside vs write-through patterns, distributed caching with Redis, CDN caching, and the pitfalls that turn your cache into a source of stale data and subtle bugs.
Caching strategies optimize data access by placing frequently used data closer to the application, reducing latency and load on upstream systems. This matters when a service experiences high read traffic, slow data sources, or expensive computation, such as database queries, external API calls, or complex analytics. A well-designed cache reduces response time from hundreds of milliseconds to sub-millisecond, turning I/O-bound operations into memory-bound ones.
The first decision is identifying which data to cache. Not all data benefits equally. Cache only data that is read-heavy, infrequently updated, and expensive to compute or retrieve.
A cache hit means the requested data is already in memory; a miss requires fetching it from the backing store. The goal is to maximize hit rate: the fraction of requests served from the cache.
The simplest and most common strategy: use a key to look up data in the cache. Keys are typically derived from input parameters, such as user:123 or post:456:comments.
GET /api/users/123This request can be cached under the key user:123. The application checks the cache for that key before querying the database.
func GetUser(id int) (*User, error) {
key := fmt.Sprintf("user:%d", id)
if cached, ok := cache.Get(key); ok {
return cached, nil
}
user, err := db.QueryRow("SELECT id, name, email FROM users WHERE id = $1", id)
if err != nil {
return nil, err
}
cache.Set(key, user, 300*time.Second) // TTL: 5 minutes
return user, nil
}The cache key must be consistent across requests. A mismatch in key construction—such as using user:123 versus user:123:—causes cache misses for identical data.
When a query depends on multiple parameters, use a composite key based on all relevant inputs.
GET /api/posts?author=456&limit=10&offset=20This query should be cached under post:author=456:limit=10:offset=20.
func GetPosts(authorID int, limit, offset int) ([]Post, error) {
key := fmt.Sprintf("post:author=%d:limit=%d:offset=%d", authorID, limit, offset)
if cached, ok := cache.Get(key); ok {
return cached, nil
}
query := `
SELECT id, title, body, author_id, created_at
FROM posts
WHERE author_id = $1
ORDER BY created_at DESC
LIMIT $2 OFFSET $3
`
rows, err := db.Query(query, authorID, limit, offset)
if err != nil {
return nil, err
}
posts := []Post{}
for rows.Next() {
var p Post
err := rows.Scan(&p.ID, &p.Title, &p.Body, &p.AuthorID, &p.CreatedAt)
if err != nil {
return nil, err
}
posts = append(posts, p)
}
cache.Set(key, posts, 60*time.Second)
return posts, nil
}This approach avoids recomputing the same query repeatedly, even with slight variations in parameters.
Caches have limited memory. When full, the system must evict old or less-used entries. Two common approaches: time-to-live (TTL) and least recently used (LRU).
Set a fixed expiration time for each cached entry. After that time, the entry is considered stale and may be evicted or refreshed.
{
"key": "user:123",
"value": {
"id": 123,
"name": "Alice",
"email": "[email protected]"
},
"expires_at": "2024-03-15T10:32:45Z"
}The cache is checked for expiration during retrieval. On a miss, the data is fetched and stored with a TTL.
TTL is simple to implement and predictable. However, it does not account for access frequency. A rarely used item may remain in the cache for hours while a heavily used one is flushed early.
Evict the least recently accessed item when the cache is full.
In practice, this is implemented using a doubly linked list where each node tracks the key and value, and the most recently accessed item is moved to the front.
When a cache miss occurs, the new item is added to the front. When the cache is full and a new entry is added, the last (least recently used) item is removed.
LRU balances between freshness and utilization. It works well when access patterns are non-uniform, but adds complexity: the cache must track access order.
Combine both: entries expire after a TTL, but are also evicted based on recency.
This is common in production. The cache checks for expiration periodically (via background cleanup), and evicts stale entries during retrieval or insertion.
When data changes, cached versions become stale. The system must decide when and how to invalidate.
On every write, update both the backing store and the cache synchronously.
func UpdateUser(user User) error {
// Update database
_, err := db.Exec("UPDATE users SET name = $1, email = $2 WHERE id = $3",
user.Name, user.Email, user.ID)
if err != nil {
return err
}
// Update cache
cache.Set(fmt.Sprintf("user:%d", user.ID), user, 300*time.Second)
return nil
}This ensures cache and database are in sync. However, it increases write latency, especially if the cache is remote (e.g., Redis).
Write to the cache first, then asynchronously update the backing store.
func UpdateUser(user User) error {
// Update cache immediately
cache.Set(fmt.Sprintf("user:%d", user.ID), user, 300*time.Second)
// Queue the write to database
go func() {
_, err := db.Exec("UPDATE users SET name = $1, email = $2 WHERE id = $3",
user.Name, user.Email, user.ID)
if err != nil {
log.Printf("Failed to update user %d in DB: %v", user.ID, err)
}
}()
return nil
}This improves write throughput but introduces eventual consistency. Reads may serve stale data until the background task completes.
The most common pattern: the application manages the cache manually.
On a read:
On a write:
This decouples cache logic from the database.
func GetUser(id int) (*User, error) {
key := fmt.Sprintf("user:%d", id)
// Try cache
if cached, ok := cache.Get(key); ok {
return cached, nil
}
// Cache miss: fetch from DB
row := db.QueryRow("SELECT id, name, email FROM users WHERE id = $1", id)
var user User
err := row.Scan(&user.ID, &user.Name, &user.Email)
if err != nil {
return nil, err
}
// Populate cache
cache.Set(key, user, 300*time.Second)
return &user, nil
}Invalidate on update:
func UpdateUser(user User) error {
_, err := db.Exec("UPDATE users SET name = $1, email = $2 WHERE id = $3",
user.Name, user.Email, user.ID)
if err != nil {
return err
}
// Invalidate cache
cache.Delete(fmt.Sprintf("user:%d", user.ID))
return nil
}This strategy is flexible and widely adopted. It allows for fine-grained control over cache behavior.
A cold cache starts with low hit rate. Warmup strategies reduce startup latency.
Load frequently accessed data into the cache during service initialization.
func Start() error {
// Preload top 100 users
rows, err := db.Query("SELECT id, name, email FROM users ORDER BY id LIMIT 100")
if err != nil {
return err
}
for rows.Next() {
var user User
err := rows.Scan(&user.ID, &user.Name, &user.Email)
if err != nil {
return err
}
cache.Set(fmt.Sprintf("user:%d", user.ID), user, 300*time.Second)
}
return nil
}This ensures high hit rates during early traffic spikes.
Predictive loading based on access patterns.
After serving a request, prefetch related data.
func GetUser(id int) (*User, error) {
key := fmt.Sprintf("user:%d", id)
if cached, ok := cache.Get(key); ok {
// Prefetch related data
go func() {
// Fetch user's posts
posts, err := GetPosts(id, 10, 0)
if err != nil {
return
}
cache.Set(fmt.Sprintf("user:%d:posts", id), posts, 60*time.Second)
// Fetch user's friends
friends, err := GetFriends(id)
if err != nil {
return
}
cache.Set(fmt.Sprintf("user:%d:friends", id), friends, 300*time.Second)
}()
return cached, nil
}
// Cache miss: fetch from DB
row := db.QueryRow("SELECT id, name, email FROM users WHERE id = $1", id)
var user User
err := row.Scan(&user.ID, &user.Name, &user.Email)
if err != nil {
return nil, err
}
cache.Set(key, user, 300*time.Second)
// Prefetch related data
go func() {
posts, err := GetPosts(id, 10, 0)
if err != nil {
return
}
cache.Set(fmt.Sprintf("user:%d:posts", id), posts, 60*time.Second)
friends, err := GetFriends(id)
if err != nil {
return
}
cache.Set(fmt.Sprintf("user:%d:friends", id), friends, 300*time.Second)
}()
return &user, nil
}Prefetching reduces latency on subsequent requests but increases memory usage and background load.
Caching introduces new failure points.
The most common failure: the requested key is not in the cache. The system falls back to the backing store.
A miss can be costly: the round-trip to the database or external service adds latency.
Data becomes outdated between cache population and expiration. A user updates their profile, but the cache serves the old version for 5 minutes.
Staleness is often accepted in read-heavy systems but requires coordination to minimize.
The cache is updated but the wrong key is invalidated.
For example, updating a user’s name, but invalidating user:123 while the actual key is user:123:profile.
This leads to inconsistent data: the user’s name is updated in the database but not in the cache.
When the cache is too small for the working set, and entries are constantly being evicted and reloaded.
This occurs when the access pattern is not localized: the application frequently jumps between unrelated data sets.
A single request with a malformed or invalid key causes the cache to store incorrect data.
For example, a query with author=123&limit=10&offset=100 returns a result, but the cache stores it under post:author=123:limit=10:offset=100—but the next request uses post:author=123:limit=10:offset=100 (with a trailing space).
This causes cache misses even when the data is already in the cache.
Caching is not limited to a single layer. Use multiple caches in series.
A small, fast cache within the application process, such as a Go map or a LruCache instance.
Used for frequently accessed, small data: user sessions, configuration, metadata.
type App struct {
cache *lru.Cache // LRU cache with 1000 entries
}This is the first line of defense: high-speed access with low latency.
A shared cache across multiple instances, such as Redis or Memcached.
Used for shared data: user profiles, product catalogs, API responses.
$ redis-cli
127.0.0.1:6379> GET "user:123"
"{"id":123,"name":"Alice","email":"[email protected]"}"This is the second line: scales across instances and survives restarts.
Use both local and distributed caches.
On a read:
This reduces latency and load on the distributed cache.
Caching is not a single technique but a set of interrelated decisions. The success of a caching strategy depends on selecting the right data, choosing effective eviction and expiration policies, and managing invalidation.
The most common failure is not realizing that a cache miss is not just a lookup—it is a full round-trip to a slow backend. The cache must be designed with that cost in mind.
A well-structured cache minimizes the number of round-trips, reduces latency, and scales under load. It transforms a system from being I/O-bound to memory-bound, serving data at the speed of memory.
This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy