Graceful Shutdown: Draining Connections Without Dropping Requests

Implement graceful shutdown patterns that let your services restart, deploy, and scale without dropping in-flight requests. Covers signal handling, connection draining, health check coordination, Kubernetes preStop hooks, and timeout strategies for zero-downtime deployments.

Graceful shutdown is the practice of stopping a service so that in‑flight work completes before the process exits. It is required whenever a host is removed from a cluster, a deployment rolls forward, a node is scaled down, or infrastructure is reclaimed. Without a graceful shutdown procedure, a load balancer or client can finish the current request and then immediately receive an error, a database transaction can be rolled back mid‑statement, or a connection pool can leave sockets in a half‑closed state. This page provides concrete steps, commands, and code patterns that let a service drain connections, let outstanding work finish, and only then exit.

Initiate the Shutdown Signal

The first step is sending the right signal and giving the process enough time to respond. Three common entry points are the operating system service manager, Docker, and Kubernetes.

A bare kill -SIGTERM <pid> is the universal equivalent; it triggers the same signal‑handling path as the tools above. The critical invariant is that SIGTERM is a request to shut down gracefully, whereas SIGKILL forcibly terminates the process, bypassing all drain logic.

Stop Accepting New Work, Finish Existing

Once the signal is delivered, the service must stop admitting new connections while completing those already in progress. The pattern is language‑agnostic: set a flag in a signal handler, break the accept/read loop, drain pools, then exit.

Python example (standard library only)

import signal, time, sys

draining = False

def _sigterm(signum, frame):
    global draining
    draining = True

signal.signal(signal.SIGTERM, _sigterm)


while not draining:
    conn, addr = sock.accept()
    # honour the flag inside the handler
    handle(conn, draining)

# after the loop: finish pending work, close listener
sock.close()
# allow remaining checked‑out connections to return/finish

The key behaviours are:

Other runtimes have analogous patterns. Node.js uses process.on('SIGTERM'), Java services register a ShutdownHook, and Go programs use os/signal to catch os.Interrupt/os.Kill. The mechanics are the same: stop accepting, let in‑flight finish, exit.

Drain the Load Balancer / Orchestrator

The infrastructure layer must stop routing new traffic to the host while it drains. The orchestrator typically handles this, but the behaviour depends on configuration.

If the load balancer continues sending requests to a host that has already signalled shutdown, clients receive 503 Service Unavailable or connection reset by peer, depending on whether the server closed the socket or merely stopped accepting.

Drain Database Connections and Connection Pools

Application‑level connection pools must also observe the drain flag. The generic pattern is:

  1. When the shutdown flag is set, the pool stops checking out new connections.
  2. Connections already checked out are allowed to finish their transactions (commit or rollback) and return to the pool.
  3. Once the pool is empty, the process exits.

PostgreSQL illustration (using pgbouncer in transaction mode, concepts apply to any pool):

Common failure mode: a process exits while the pool still holds checked‑out connections. The orchestrator kills the pod (SIGKILL), the pool is abandoned, and the database logs could not receive data from client. The client receives connection reset by peer. To avoid this, always ensure the application’s signal handler sets the drain flag *before* the orchestrator’s grace period expires.

Verify the Shutdown Completed

After the signal, drain, and exit, confirm that the host is fully out of the loop.

Sharp Edges – What Silently Fails

Even with the pattern above, production outages occur when these edge cases are overlooked.

Understanding these edges before they manifest in production is far less costly than debugging dropped requests, timed‑out connections, or database errors after a shutdown has already begun.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.