Websocket Scaling Guide

Production-grade guide to websocket scaling guide covering architecture patterns, implementation strategies, testing approaches, and operational best practices for enterprise engineering teams.

Websockets enable real-time, bidirectional communication between clients and servers. This guide details how to scale websockets across multiple backend instances, ensuring low latency, high throughput, and resilience under load. It is essential when a system supports live dashboards, chat applications, collaborative editing, or real-time monitoring. Scaling is not optional when clients expect immediate updates across thousands of concurrent connections.

Establishing the Foundation

A basic websocket setup involves a single server handling all connections. As demand grows, the server becomes a bottleneck—memory usage rises, CPU saturates, and new clients experience latency. The first step is to scale horizontally by deploying multiple instances of the backend, each serving clients. However, this introduces a new challenge: how does a message sent to one instance reach all connected clients, even those on other instances?

To solve this, a shared message broker becomes the central nervous system. All backend instances connect to the broker to publish and subscribe to events. The broker receives messages from any instance and forwards them to all clients, ensuring global visibility.

Setting Up a Shared Message Broker

Use Redis as the message broker. It is widely adopted, performant, and supports pub/sub natively.

Start the Redis server:

docker run -d --name redis -p 6379:6379 redis:7

In your backend code, connect each instance to Redis and use it for broadcasting. For example, in Python with aioredis:

import asyncio
import aioredis

async def broadcast_message(channel: str, message: str):
    redis = await aioredis.from_url("redis://localhost:6379")
    await redis.publish(channel, message)
    await redis.close()

When a client connects, the server subscribes to a channel like chat:room:123. Any message sent to that channel is pushed to all clients subscribed to it.

Handling Client Connections Across Instances

Each backend instance maintains its own set of open websocket connections. To route incoming messages to the correct client, use connection metadata stored in Redis.

For each connected client, store metadata in a Redis hash:

HSET client:12345 \
    user_id "alice" \
    room_id "123" \
    last_seen "2025-04-05T10:30:00Z" \
    session_id "abc123"

When a client sends a message, the server:

  1. Reads the session_id from the websocket frame.
  2. Looks up the client in Redis to determine which backend instance owns the connection.
  3. Sends the message to that instance via a message queue or direct RPC.

To ensure clients are routed correctly, implement session affinity using Redis as a session store.

Distributing Workload with Load Balancing

A load balancer directs incoming websocket connections to backend instances. The key is sticky sessions (session persistence), so each client consistently connects to the same server.

The load balancer must:

Use Nginx as the load balancer. Configure it to handle WebSocket upgrades and maintain session affinity via cookies.

upstream backend {
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
}

server {
    listen 443 ssl;
    server_name chat.example.com;

    location /ws {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Sticky sessions using session cookie
        proxy_cookie_path / "/; secure; HttpOnly; SameSite=Strict";
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;

        # Keepalive for long-lived connections
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_cache_bypass $http_upgrade;
    }
}

After establishing a WebSocket connection, the client receives a Set-Cookie header:

Set-Cookie: session_id=abc123; Path=/; Secure; HttpOnly; SameSite=Strict

The load balancer uses this cookie to route future frames to the same backend instance.

Broadcasting Messages to All Clients

When a user sends a message in a chat room, the backend must deliver it to every client in that room. With multiple instances, this requires distributed broadcasting.

Each instance publishes to a Redis channel, and a central message bus ensures all clients receive it.

For a message in room 123, the steps are:

  1. Backend instance receives the message.
  2. It publishes {"event": "chat.message", "data": {"text": "Hello", "sender": "alice"}} to chat:room:123.
  3. Redis broadcasts the message to all subscribers.
  4. Every client connected to any instance receives the update.

To ensure messages are delivered even when instances restart, use Redis Pub/Sub with persistent subscriptions.

Ensuring Delivery During Instance Failures

If a backend instance crashes, clients it hosted lose connection. To recover:

To avoid missing messages during the outage, message replay is critical.

When a client reconnects, it sends its last_seen timestamp. The server queries Redis to find all messages sent after that timestamp.

async def get_messages_since(client_id: str, since: str):
    redis = await aioredis.from_url("redis://localhost:6379")
    messages = await redis.zrangebyscore(
        f"messages:client:{client_id}",
        min=since,
        max="+inf"
    )
    await redis.close()
    return messages

Clients send a replay message after connecting:

{
  "type": "replay",
  "last_seen": "2025-04-05T10:28:00Z"
}

The server replies with all missed messages.

Scaling the Message Broker

As the number of messages increases, Redis becomes a bottleneck. It must handle thousands of publish operations per second and maintain low latency.

The primary failure mode: Redis becomes the single point of contention. When the broker is slow, all instances pause during message delivery.

To detect this, monitor Redis with redis-cli:

redis-cli --latency -i 1

This shows average round-trip time between client and Redis. A latency spike indicates the broker is under load.

Use Redis Streams to manage message queues efficiently.


XADD messages:chat:123 * event chat.message data '{"text":"Hi","sender":"bob"}'

Each instance can consume from the stream:

import aioredis

async def consume_messages():
    redis = await aioredis.from_url("redis://localhost:6379")
    while True:
        messages = await redis.xread(
            {"messages:chat:123": "0"},
            block=1000,  # 1 second
            count=100
        )
        for stream, entries in messages:
            for entry_id, fields in entries:
                await handle_message(fields)

This allows asynchronous, high-throughput message processing.

Handling Thousands of Concurrent Connections

Each backend instance can handle up to 10,000 concurrent websockets. Beyond that, memory usage grows rapidly, and GC pauses become noticeable.

To tune for high concurrency, adjust the event loop settings and socket options.

In Node.js, use cluster to spawn multiple worker processes, each with its own event loop.

const cluster = require('cluster');
const http = require('http');

if (cluster.isMaster) {
    // Fork workers
    for (let i = 0; i < require('os').cpus().length; i++) {
        cluster.fork();
    }

    cluster.on('exit', (worker) => {
        console.log(`Worker ${worker.process.pid} died`);
        cluster.fork();
    });
} else {
    const server = http.createServer((req, res) => {
        res.writeHead(200, { 'Content-Type': 'text/plain' });
        res.end('Hello from worker ' + process.pid);
    });

    const WebSocket = require('ws');
    const wss = new WebSocket.Server({ server });

    wss.on('connection', (ws, req) => {
        ws.on('message', (data) => {
            wss.clients.forEach((client) => {
                if (client !== ws) {
                    client.send(data);
                }
            });
        });
    });

    server.listen(8080);
}

Each worker process runs its own event loop and manages its own set of connections.

To reduce memory pressure, compress messages using MessagePack or Protobuf instead of JSON.

Memory Pressure and GC Pauses

When the number of open connections grows, the garbage collector on each instance starts pausing the application.

Monitor with node --inspect and chrome://inspect:

node --inspect server.js

In the Chrome DevTools, examine the Memory tab for:

A common failure: memory leaks in WebSocket handlers.

A client sends a subscribe message:

{
  "type": "subscribe",
  "channel": "user:alice"
}

But the server forgets to remove the client from the subscription list when the connection closes.

Fix by using WebSocket close handlers:

ws.on('close', () => {
    const index = subscriptions[user_id].indexOf(ws);
    if (index > -1) {
        subscriptions[user_id].splice(index, 1);
    }
});

Also, set idle timeout and ping-pong to detect dead connections.

Ping-Pong Heartbeats

Set up periodic pings from the server to clients.

In Node.js with ws:

const pingInterval = setInterval(() => {
    wss.clients.forEach((client) => {
        if (client.readyState === WebSocket.OPEN) {
            client.ping();
        }
    });
}, 30000); // 30 seconds

Clients respond with pong. If a client fails to respond within 60 seconds, the server marks it as disconnected.

Handling Message Ordering and Deduplication

When clients send messages rapidly, messages may arrive out of order, especially after reconnecting.

Use message IDs to ensure ordering and deduplication.

Each message includes a message_id and timestamp:

{
  "message_id": "msg-12345",
  "timestamp": "2025-04-05T10:30:00Z",
  "type": "chat.message",
  "data": { "text": "Hello", "sender": "alice" }
}

The server stores received messages in Redis with message_id as the key:

SET message:msg-12345 '{"type":"chat.message", "data":{...}}'

When a client reconnects, it sends a last_message_id in its reconnect message:

{
  "type": "reconnect",
  "last_message_id": "msg-12345"
}

The server returns all messages with message_id greater than msg-12345.

To prevent duplicate messages, use idempotency.

When a client sends a message, it includes a request_id:

{
  "request_id": "req-abc123",
  "type": "chat.message",
  "data": { "text": "Hi", "sender": "alice" }
}

The server checks if request_id exists before processing the message. If yes, it returns the previously processed result without reprocessing.

Failure Modes and Monitoring

Silent Failures

Monitor with:

Common Configuration Pitfalls

Critical Metrics

Recovery Commands

When a backend instance fails:

# Restart the service
systemctl restart backend

# Check logs
journalctl -u backend -f --since "1 hour ago"

# Force message replay for all clients
redis-cli EVAL "
    local clients = redis.call('KEYS', 'client:*')
    for _, client_id in ipairs(clients) do
        local last_seen = redis.call('HGET', client_id, 'last_seen')
        redis.call('PUBLISH', 'replay:client:' .. client_id, last_seen)
    end
" 0

This script triggers replay for every client in Redis.

Summary

Scaling websockets requires attention to message delivery, instance coordination, and connection resilience. Use Redis as a shared message broker. Configure Nginx for sticky sessions. Implement message replay and idempotency. Monitor connection counts, message rates, and latency. Address failure modes like dropped connections, memory leaks, and out-of-order messages. With these patterns, systems handle thousands of concurrent clients with predictable performance and minimal downtime.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.