Request Coalescing

Request coalescing is a backend pattern for reducing redundant network work by merging multiple similar incoming requests into a single, shared computation. It matters when multiple clients or services issue overlapping or identical queries in rapid succession—such as a user dashboard loading simultaneously across tabs, or a microservice fetching user profiles for a batch of IDs—and the underlying data source is expensive to compute, I/O-bound, or both.

When to Coalesce Requests

Coalescing is most valuable when the cost of executing a request exceeds the cost of coordinating the coalescing logic. This typically occurs when:

• The request involves a remote database query with high latency.
• The data is computed via a complex, stateful process (e.g., aggregating logs, recomputing recommendations).
• The service is under high load with many concurrent clients.
• The response is large and expensive to serialize and transmit.

For instance, if ten clients request the same user profile within 200ms of each other, coalescing ensures only one database query runs, serving all ten requests with a single result.

How Coalescing Works: The Core Pattern

At its heart, request coalescing involves a central coordinator that tracks incoming requests and delays execution until a window of time or a threshold of requests is met. The coordinator maintains a registry of pending requests, keyed by a unique identifier derived from the request's parameters.

The coordinator proceeds in three phases:

1. Request Arrival: A client sends a request. The coordinator checks if a matching request is already in progress.
2. Coalescing: If a matching request exists, the new request joins the existing one’s wait list. If not, the coordinator starts a new request and begins a timer.
3. Execution & Response: Once the coalescing window expires or the request count reaches a threshold, the coordinator executes the underlying computation. All waiting clients receive the same result.

The key is that the identifier used to group requests must be stable and representative of the request’s intent.

Request Keying: Defining What Makes Requests “the Same”

Requests are coalesced based on a request key, a canonical representation of the request’s parameters. The most common approach uses a hash of the request’s parameters, typically including:

• HTTP method
• Path
• Query parameters
• Headers (especially those affecting the computation)
• Request body (for POST/PUT)

For example, a request to /api/users/123?include=posts&format=compact and a request to /api/users/123?include=posts&format=compact&locale=en should be coalesced if the format and locale headers are part of the key.

In practice, request keying often involves a function like:

func RequestKey(r *http.Request) string {
    var buf strings.Builder
    buf.WriteString(r.Method)
    buf.WriteString(r.URL.Path)
    query := r.URL.Query()
    keys := make([]string, 0, len(query))
    for k := range query {
        keys = append(keys, k)
    }
    sort.Strings(keys)

    for _, k := range keys {
        buf.WriteString(k)
        for _, v := range query[k] {
            buf.WriteString(v)
        }
    }

    // Include headers that affect behavior
    for _, h := range []string{"Accept", "Content-Type", "X-User-ID"} {
        if value := r.Header.Get(h); value != "" {
            buf.WriteString(h)
            buf.WriteString(value)
        }
    }

    // Include body if present
    if r.Body != nil {
        body, _ := io.ReadAll(r.Body)
        buf.Write(body)
    }

    return sha256.Sum256([]byte(buf.String()))
}

This function produces a unique key for each request. The coordinator uses this key to match incoming requests.

Coalescing Configuration: Timing and Thresholds

Coalescing is governed by two parameters: timeout and threshold.

• Timeout: The maximum time to wait for new requests before executing the coalesced batch.
• Threshold: The minimum number of requests that must be coalesced before execution.

These parameters are configurable and often applied per endpoint or service.

A common configuration in a JSON-based config file (e.g., config.yaml):

coalescing:
  - endpoint: "/api/users/{id}"
    timeout: 100ms
    threshold: 3
    cache_ttl: 1s
    max_concurrent: 100
  - endpoint: "/api/orders"
    timeout: 250ms
    threshold: 5
    cache_ttl: 5s
    max_concurrent: 50

This config instructs the system to:

• Coalesce all requests to /api/users/{id} into a single query if at least three requests arrive within 100ms.
• Coalesce requests to /api/orders when five requests arrive within 250ms.
• Cache the result for the duration of the coalescing window.

The cache_ttl field determines how long the computed result is available for subsequent requests, even after the coalescing window has closed.

Request Coalescing in Practice: A Step-by-Step Example

Consider a service that computes user profiles. Clients send requests like:

GET /api/users/42?include=posts&format=compact HTTP/1.1
Host: api.example.com
Accept: application/json
X-User-ID: 100

The coalescing coordinator starts with a timeout of 100ms and threshold of 3.

1. T = 0ms: Client A sends request → no matching request exists → coordinator starts new coalescing job with key GET:/api/users/42?include=posts&format=compact.
2. T = 30ms: Client B sends same request → key matches → B joins the job → now 2 requests in queue.
3. T = 60ms: Client C sends same request → key matches → C joins → now 3 requests → threshold reached.
4. T = 65ms: Coordinator triggers execution: runs query against database → returns result.
5. T = 70ms: All three clients receive same response.
6. T = 120ms: Client D sends request → new coalescing job starts.
7. T = 140ms: Client E sends same request → E joins the job.
8. T = 190ms: Client F sends same request → F joins → now 3 requests → threshold reached.
9. T = 195ms: Coordinator executes → new result sent to D, E, F.

If no new requests arrive by T = 200ms, the job completes. If a new request arrives at T = 205ms, it starts a new job.

Common Pitfalls and Failure Modes

1. Keying Too Coarse or Too Fine

• Too coarse: Requests with different locale or format headers are not coalesced, even though they could be.
• Too fine: Requests with slightly different query parameters (e.g., ?include=posts vs ?include=posts&) are not merged.

A frequent mistake is forgetting to include a header that significantly affects the computation. For example, a X-User-ID header that influences caching and query logic is often omitted from the key, leading to missed coalescing.

2. Stale Results from Overlapping Windows

When coalescing windows overlap, the result from an earlier window may be outdated by the time a later window starts.

For instance, if the database is updated between T = 100ms and T = 150ms, and a new coalescing job starts at T = 130ms, the result from the first job may not reflect the latest data.

This leads to stale coalescing results—clients receive responses computed from outdated data.

Solutions include:

• Using timestamped keys that include the time of query execution.
• Applying a versioned cache where each coalesced result has a version number tied to the database state.

3. Request Timeout Misalignment

If the coalescing timeout is longer than the client’s request timeout, clients may time out before receiving a response.

For example, a client has a 500ms timeout, but the coalescing window is 250ms. If the coordinator starts execution at T = 200ms, the client may time out at T = 500ms, but the response arrives at T = 300ms—so the client waits an extra 200ms.

To mitigate this:

• Set the coalescing timeout to be shorter than the client’s expected timeout.
• Return a 202 Accepted response immediately upon request arrival, with a Location header pointing to the final result.

HTTP/1.1 202 Accepted
Location: /api/users/42?etag=abc123
Content-Length: 0

Clients can poll the Location URI for the final result.

4. Heartbeat and Cleanup of Idle Coalescing Jobs

A coalescing job that starts at T = 0ms but receives no new requests until T = 1000ms may be idle for 900ms. If the service is under low load, this job may be wasted.

To avoid this:

• Implement heartbeat checks that prune idle jobs.
• Use a timer per job, which fires when the timeout is reached or when a new request arrives.

Jobs that have not received a new request for longer than timeout * 1.5 may be considered “stale” and can be merged or preempted.

5. Caching the Final Result

After execution, the coalesced result is typically cached. The cache key includes the request key and the execution timestamp.

A common mistake is forgetting to invalidate the cache when the underlying data changes.

For example, if a user’s profile is updated after a coalescing job finishes, clients may receive outdated data.

Solutions:

• Use cache-invalidation signals (e.g., via Redis pub/sub or Kafka events).
• Set a TTL that accounts for expected data staleness.
• Include a cache version in the response header:

ETag: "v2-user-42-2024-05-15T10:00:00Z"
Cache-Control: public, max-age=300, s-maxage=600

Clients can use If-None-Match to fetch updated results.

Integration with Existing Systems

Request coalescing works well with:

• API gateways that can route requests and track state.
• Message queues (e.g., RabbitMQ, Kafka) to coordinate execution across services.
• Caching layers (e.g., Redis, Memcached) for storing coalesced results.

In a typical deployment:

1. An API gateway receives requests and forwards them to a coalescing service via HTTP or gRPC.
2. The coalescing service maintains in-memory state (e.g., using Go maps or Redis) for active jobs.
3. When a job is ready, it invokes a backend service (e.g., via curl) to compute the result:

curl -X POST \
     -H "Content-Type: application/json" \
     -d '{"user_id": 42, "include": ["posts"], "format": "compact"}' \
     http://profile-service/api/compute-profile

1. The result is stored in Redis and returned to all waiting clients.

Error Handling and Failure Propagation

When the coalescing execution fails, the coordinator must propagate the error to all waiting clients.

Failure modes include:

• Database connection lost during execution.
• Service unavailable when computing result.
• Timeout in the underlying computation.

The coordinator returns a 503 Service Unavailable with a payload:

{
  "error": "computation_failed",
  "message": "Failed to compute user profile for user 42",
  "details": {
    "error_code": "DB_TIMEOUT",
    "timestamp": "2024-05-15T10:00:00Z",
    "retry_after": 5000
  }
}

Clients can:

• Retry the request.
• Fall back to a cached version.
• Show a user-friendly error.

If the failure is transient, the coordinator may replay the failed job, merging it into a new window.

Summary

Request coalescing is a powerful, predictable pattern for reducing redundant work in high-load systems. Its success depends on careful request keying, appropriate timeout and threshold configuration, and robust error handling. When implemented correctly, it delivers significant performance gains with minimal infrastructure. But its subtleties—stale results, keying mismatches, and timing drifts—are often the source of silent degradation in production.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.