Websocket Scaling

Websocket connections enable real-time, bidirectional communication between clients and servers. They are essential when applications require immediate updates—live dashboards, collaborative editing, chat systems, and IoT telemetry—where polling or REST APIs fall short. Scaling websockets across multiple server instances, handling thousands of concurrent connections, and ensuring message delivery across instance boundaries are common challenges. This reference page details how to build and operate a robust, scalable websocket system, from initial setup to diagnosing and resolving common failures.

Connection Management

A websocket connection begins with an HTTP upgrade handshake. Clients send an Upgrade: websocket header, and the server responds with a 101 Switching Protocols status. The connection remains open, enabling full-duplex communication.

To manage connections across multiple backend instances, use a load balancer that supports sticky sessions (session affinity). The load balancer must preserve the client-server pairing across the lifetime of a websocket. For TCP-based load balancers, this is achieved via the X-Forwarded-For and X-Real-IP headers, and the Connection header must be preserved during the upgrade.

For example, in NGINX, enable sticky sessions using the ip_hash directive:

upstream websocket_backend {
    ip_hash;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

server {
    listen 80;
    location /ws {
        proxy_pass http://websocket_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering off;
        proxy_cache off;
    }
}

The proxy_buffering off and proxy_cache off directives prevent the NGINX buffer from delaying message delivery. Without them, messages may be delayed by up to 10 seconds due to buffering.

When a client connects to instance 10.0.1.10, all messages sent from the client are routed to that instance. However, if the instance fails or is restarted, the client may lose the connection. To avoid this, use a session-aware load balancer, or implement a connection migration strategy using a shared state store.

Message Distribution Across Instances

When multiple backend instances serve the same websocket clients, messages sent from one instance must reach clients connected to other instances. This requires a message broadcast mechanism.

Use a message broker such as Redis, RabbitMQ, or Apache Kafka. Messages are published to a topic or queue, and all instances subscribe to that topic.

In a Go-based backend using the gorilla/websocket library, a typical setup involves:

import (
    "context"
    "github.com/gorilla/websocket"
    "github.com/go-redis/redis/v8"
)

type WebsocketManager struct {
    clients map[string]*Client
    broadcast chan []byte
    rdb *redis.Client
    ctx context.Context
}

func (wm *WebsocketManager) Start() {
    wm.broadcast = make(chan []byte, 100)
    wm.rdb = redis.NewClient(&redis.Options{
        Addr: "redis:6379",
        Password: "",
        DB: 0,
    })
    wm.ctx = context.Background()

    // Subscribe to Redis channel for broadcast
    pubsub := wm.rdb.Subscribe(wm.ctx, "broadcast")
    go func() {
        for msg := range pubsub.Channel() {
            wm.broadcast <- []byte(msg.Payload)
        }
    }()

    // Start broadcast loop
    go wm.broadcastMessages()
}

func (wm *WebsocketManager) broadcastMessages() {
    for msg := range wm.broadcast {
        for _, client := range wm.clients {
            if err := client.Conn.WriteMessage(websocket.TextMessage, msg); err != nil {
                // Handle client error
                client.Conn.Close()
                delete(wm.clients, client.ID)
            }
        }
    }
}

Each server instance publishes messages to the broadcast Redis channel. Clients are identified by a unique ID, such as a UUID or a session token. When a client sends a message, the server writes it directly to the client’s connection and also publishes it to the Redis channel.

This setup ensures that a message sent from instance 10.0.1.10 is received by all clients, regardless of which instance they are connected to. If instance 10.0.1.11 fails, clients connected to it continue receiving messages from other instances, and messages published from 10.0.1.10 are delivered to all clients.

State Synchronization and Reconnection

Clients reconnect after network interruptions or server restarts. During reconnection, they must restore their state: which channels they are listening to, the current view of the data, and any pending messages.

Use a client-side reconnection strategy based on exponential backoff with jitter. The client retries connection attempts with increasing delays, and includes a Last-Message-ID or Sequence-Number header to resume from the last received message.

On the server, maintain a state store (e.g., PostgreSQL or Redis) that tracks:

• Client metadata (user ID, session ID, client type)
• Subscribed channels per client
• Last-seen message IDs
• Active connection IDs

When a client reconnects, the server:

1. Reads the client’s state from the store.
2. Sends a reconnect event with the client’s last-seen message ID.
3. Sends all messages from that ID onward.
4. Resumes streaming from the point of failure.

Example of a client reconnection sequence:

{
  "type": "reconnect",
  "lastMessageId": 12345,
  "sessionId": "abc123",
  "clientId": "client-8899"
}

The server responds with a list of messages and a joined event for each channel the client was subscribed to.

If the client reconnects while a message is being published, the message may arrive before the reconnection is complete. To avoid duplication, use message IDs and deduplication.

Load Balancer Health Checks and Instance Lifecycle

The load balancer must detect failed backend instances and route traffic accordingly. Use HTTP health checks with a /health endpoint.

func healthHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(map[string]interface{}{
        "status": "ok",
        "uptime": time.Since(startTime).Seconds(),
        "websocketConnections": len(websocketManager.Clients),
        "pendingMessages": len(websocketManager.Broadcast),
    })
}

The load balancer pings this endpoint every 10 seconds. If a server fails three consecutive checks, it is removed from the pool.

When a new instance starts, it:

1. Registers with the message broker (e.g., Redis).
2. Subscribes to the broadcast channel.
3. Joins the health check pool.
4. Starts serving new clients.

If the instance crashes during message processing, the message broker ensures no messages are lost. Redis or RabbitMQ persist messages across instance restarts.

Failure Modes and Diagnostics

Silent Connection Drops

Clients may appear connected but fail to receive messages. Common causes:

• TCP keep-alive not configured.
• Load balancer timeout (e.g., 30 seconds) while the backend instance is processing a long task.

To detect this, enable TCP keep-alive on the server:

// In Go, after accepting a connection
conn, err := listener.Accept()
if err != nil {
    log.Println("Error accepting connection:", err)
    return
}

// Set TCP keep-alive
tcpConn, ok := conn.(*net.TCPConn)
if !ok {
    log.Println("Connection is not a TCP connection")
    return
}

tcpConn.SetKeepAlive(true)
tcpConn.SetKeepAlivePeriod(30 * time.Second)

// Set TCP keep-alive probes
tcpConn.SetKeepAlivePeriod(30 * time.Second)
tcpConn.SetKeepAliveCount(3)
tcpConn.SetKeepAliveIdle(10 * time.Second)

This ensures the connection remains open even if no messages are sent.

Message Loss on Instance Failure

When an instance fails, messages it was processing may be lost if they are not yet published to the message broker.

To prevent this, use a two-phase commit:

1. The instance writes the message to its in-memory buffer.
2. It publishes the message to the broker.
3. It acknowledges receipt to the client.
4. Only then does it process the message.

If the instance crashes before publishing, the message is not lost—it is picked up by other instances during reconnection.

Overlapping Broadcasts and Duplicate Messages

When multiple instances publish to the same channel, messages may arrive in overlapping batches. Clients may process the same message twice.

Use message deduplication via a unique message_id field in the payload:

{
  "message_id": "msg-1234567890",
  "timestamp": "2023-08-24T10:00:00Z",
  "type": "notification",
  "data": { "title": "New post", "id": 45 }
}

The client maintains a set of seen message_ids. When a message arrives, it checks if the ID is already in the set. If so, it skips processing.

Configuration for Production

Use environment variables for key settings:

Variable	Value Example	Description
REDIS_HOST	redis:6379	Redis server address
WEBSOCKET_PORT	8080	Listening port for websocket server
HEALTH_CHECK_INTERVAL	10s	Interval between health checks
KEEP_ALIVE_INTERVAL	30s	TCP keep-alive interval
MAX_CONNECTIONS	10000	Maximum number of concurrent connections
BROADCAST_BUFFER_SIZE	1000	Size of broadcast message buffer

Set these in a docker-compose.yml:

version: '3.8'

services:
  backend:
    build: .
    ports:
      - "8080:8080"
    environment:
      - REDIS_HOST=redis:6379
      - WEBSOCKET_PORT=8080
      - HEALTH_CHECK_INTERVAL=10s
      - KEEP_ALIVE_INTERVAL=30s
      - MAX_CONNECTIONS=10000
      - BROADCAST_BUFFER_SIZE=1000
    depends_on:
      - redis
    restart: unless-stopped

  redis:
    image: redis:7
    ports:
      - "6379:6379"
    command: ["redis-server", "--appendonly", "yes"]
    volumes:
      - redis_data:/data
    restart: unless-stopped

volumes:
  redis_data:

The --appendonly yes flag ensures Redis persists all writes to disk, reducing data loss on instance failure.

Monitoring and Logging

Log key events at the following levels:

• INFO: client connected, message received, client disconnected.
• WARN: client reconnected, message deduplication, heartbeat timeout.
• ERROR: failed message publish, client write error, health check failure.

Use structured logging with JSON output:

{
  "timestamp": "2023-08-24T10:00:00Z",
  "level": "INFO",
  "message": "Client connected",
  "client_id": "client-8899",
  "user_id": "user-123",
  "ip": "192.168.1.100",
  "channel": "dashboard"
}

Set up a monitoring dashboard using systemctl logs, journalctl, and prometheus metrics:

• ws_connections_total: total number of connections.
• ws_messages_sent_total: total messages sent.
• ws_message_latency_ms: time from publish to delivery.
• ws_client_reconnects_total: number of reconnection events.

Use curl to test endpoint availability:

curl -v http://localhost:8080/health

Expect a 200 status and a JSON response with status: ok.

Summary

Scaling websockets requires coordinating connection state across multiple instances, distributing messages reliably, and handling reconnections gracefully. The core pattern involves:

• Sticky load balancing with TCP keep-alive.
• A message broker for broadcasting.
• A shared state store for client state.
• Robust reconnection with deduplication.
• Health checks and monitoring.

Failures occur silently: dropped connections, lost messages, duplicated events. The key to resilience is using a consistent, observable protocol—based on HTTP, TCP, and message queues—and configuring the system with concrete, real-world values.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.