Network Capacity Planning

Production engineering guide for network capacity planning covering patterns, implementation strategies, and operational best practices.

Network capacity planning is the process of sizing network infrastructure—bandwidth, routing, and transport—to meet current and projected traffic demands, ensuring that latency, throughput, and reliability remain within acceptable bounds under expected load. It matters when an application’s performance degrades under user growth, during peak traffic, or after a major deployment, and when the cost of over-provisioning or under-provisioning is visible in service outages or wasted resources.

Assessing Current Traffic Patterns

To begin capacity planning, first gather data on actual traffic across the network. Use packet capture, flow data, and monitoring to extract volume, distribution, and behavior.

Capture traffic continuously using tcpdump with a persistent, rotating file setup:

tcpdump -i eth0 -B 64k -C 100M -W 10 -w /var/captures/traffic-%Y%m%d-%H%M.pcap -W 10 -G 3600 -Z root

This command runs on a core router or edge server, writing 100MB files every hour, rotating up to 10 files, and using a 64KB buffer for efficient packet buffering.

Export flow data using sflowtool to collect per-connection metrics:

sflowtool -i 10 -d 5 -l 100000 -t 1000000

This aggregates flows every 10 seconds, with 5-second interval updates, and writes to stdout, where it can be piped into a collector.

Analyze flow data with awk to compute per-IP and per-port traffic:

sflowtool -i 10 -d 5 -l 100000 -t 1000000 | \
  awk '{
    src = $1; dst = $2; src_port = $3; dst_port = $4;
    bytes = $10; packets = $11;
    sum_bytes[src, dst] += bytes; sum_pkts[src, dst] += packets;
    count[src, dst]++;
  }
  END {
    for (key in sum_bytes) {
      split(key, parts, SUBSEP);
      src = parts[1]; dst = parts[2];
      printf "%s -> %s: %d bytes, %d packets (%d flows)\n",
             src, dst, sum_bytes[key], sum_pkts[key], count[key];
    }
  }'

This generates a summary of traffic volume and flow count by endpoint pair, revealing bottlenecks and traffic hotspots.

Estimating Peak Load and Growth

Use historical data to project future demand. Compute daily and hourly averages, then identify the 95th percentile of traffic over a week.

Extract peak traffic from a log of hourly bandwidth usage:


grep "2024-05" /var/log/bandwidth-2024.log | \
  awk '{
    split($1, t, "T");
    hour = substr(t[2], 1, 2);
    total_bytes += $2;
    total_packets += $3;
    hour_bytes[hour] += $2;
    hour_count[hour]++;
  }
  END {
    for (h = 0; h < 24; h++) {
      if (hour_bytes[h] == 0) {
        hour_bytes[h] = hour_bytes[h-1];
      }
    }
    n = asort(hour_bytes, sorted);
    print "95th percentile hourly bandwidth:", sorted[int(0.95 * n)];
  }'

This computes the 95th percentile of hourly bandwidth, giving the traffic level exceeded only 5% of the time.

To estimate growth, apply a compound growth model based on weekly data:

# Input: weekly total bytes per day (7 lines)
awk '
  BEGIN {
    total = 0;
    for (i = 0; i < 7; i++) {
      getline; total += $1;
      daily[i] = $1;
    }
    mean = total / 7;
    for (i = 0; i < 7; i++) {
      deviation[i] = daily[i] - mean;
    }
    # Compute weekly growth rate
    growth = 1;
    for (i = 1; i < 7; i++) {
      growth *= daily[i] / daily[i-1];
    }
    growth_rate = (growth) ** (1/6);
  }
  {
    # Project 12 weeks ahead
    for (week = 1; week <= 12; week++) {
      projected = daily[6] * (growth_rate ** week);
      printf "Week %d: %d bytes\n", week, projected;
    }
  }'

This computes a weekly growth factor and projects traffic 12 weeks into the future, useful for planning upgrades.

Validating Latency and Throughput Constraints

Define the service-level requirements: acceptable latency, packet loss, and throughput under load.

Use ping with fixed interval and large packets to measure baseline round-trip time and jitter:

ping -c 1000 -i 0.2 -s 1472 -t 128 -W 2000 -n 192.168.1.10

This sends 1,000 ICMP packets every 200ms, each 1,472 bytes (including header), with a 2-second timeout, and outputs statistics including min, max, average, and jitter.

Use iperf3 to measure actual throughput between two endpoints:

# Server side
iperf3 -s -p 5201 -D

# Client side
iperf3 -c 192.168.1.10 -p 5201 -t 60 -i 10 -b 100M -l 131072 -P 8 -u

This runs a 60-second UDP test with 8 parallel streams, each sending at 100 Mbps, using 128KB packets. The -u flag enables UDP mode.

Analyze the output:

------------------------------------------------------------
Client connecting to 192.168.1.10, port 5201
Sending 8 parallel streams
Starting test: 60 seconds
------------------------------------------------------------
[ ID] Interval           Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[  4]   0.00-10.00 sec  1.12 GBytes  964.3 Mbits/sec  0.012 ms  1234/132000 (0.93%)
[  5]   0.00-10.00 sec  1.13 GBytes  971.2 Mbits/sec  0.015 ms  1123/132000 (0.85%)
...
[ 11]   0.00-10.00 sec  1.11 GBytes  950.4 Mbits/sec  0.010 ms  1345/132000 (1.02%)
[SUM]   0.00-10.00 sec  8.98 GBytes  7.70 Gbits/sec  0.013 ms  8580/1056000 (0.81%)

The Bandwidth column shows the actual throughput per stream, and Jitter and Lost/Total indicate reliability. Use this data to validate whether current links can support the projected load.

Sizing Network Infrastructure

Determine the required capacity of links, routers, switches, and edge services based on peak load and constraints.

For a core router, calculate required link speed:

# Total expected traffic in Mbps
total_traffic = 15_000_000_000  # bytes per day
traffic_per_second = total_traffic / (24 * 60 * 60)  # ≈ 173.6 Mbps

With 95th percentile traffic at 173.6 Mbps, apply a 20% margin for burst and growth:

required_bandwidth = 173.6 * 1.2  # ≈ 208.3 Mbps

Choose a link speed: 250 Mbps (Gigabit Ethernet, 1 Gbps) provides headroom. For high-availability, use dual 1 Gbps links in active-active mode with link aggregation.

Configure link aggregation using LACP (Link Aggregation Control Protocol):

# On Linux with bonding
cat > /etc/bonding.conf << 'EOF'
# Bond interface bond0
mode=4
miimon=100
xmit_hash_policy=layer2+3
primary=eth0
primary_reselect=always
EOF

# Apply configuration
modprobe bonding
echo "bond0" > /etc/modules-load.d/bonding.conf
systemctl enable networking

Then assign interfaces:

# /etc/network/interfaces
auto bond0
iface bond0 inet static
    address 192.168.1.100
    netmask 255.255.255.0
    gateway 192.168.1.1
    bond-mode 4
    bond-miimon 100
    bond-xmit-hash-policy layer2+3
    bond-primary eth0
    bond-primary-reselect always

auto eth0
iface eth0 inet manual
    bond-master bond0

auto eth1
iface eth1 inet manual
    bond-master bond0

This setup enables automatic failover, load balancing, and improved throughput.

Planning for Redundancy and Failure Modes

Design for failure by identifying single points of failure and planning recovery.

Use ping with failure detection and automatic failover:

# Run a heartbeat test every 2 seconds
ping -i 2 -w 10 -c 100 -f -t 300 192.168.1.1

# If loss exceeds 5%, trigger failover
if [ $(ping -c 10 -i 2 -w 10 -f -t 300 192.168.1.1 | awk '/packet loss/,/---/' | tail -n1 | cut -d' ' -f4 | tr -d '%' | awk '{print int($1)}') -gt 5 ]; then
  systemctl restart keepalived
fi

This script monitors the gateway and restarts the failover daemon if packet loss exceeds 5%.

For data centers, plan for link failures using BGP with route advertisement and health checks.

Announce a route with a specific prefix:

# BGP peer with IP 192.168.1.1
bgp peer 192.168.1.1 {
    remote-as 65001
    update-source eth0
    advertisement interval 30
    route-refresh
    hold-time 180
    keepalive 60
    local-preference 100
    default-originate
    network 192.168.100.0/24
    network 192.168.101.0/24
}

This configures a BGP peer with a 3-minute hold time and announces two subnets.

Track route reachability using bgpctl:

bgpctl show neighbors
bgpctl show routes 192.168.100.0/24
bgpctl show summary

These commands report neighbor status, advertised routes, and summary statistics.

Failures often arise silently: BGP sessions drop due to MTU mismatches, where a 1,500-byte packet is fragmented, but the path does not support jumbo frames.

Check for MTU mismatches:

# Send a 1,500-byte packet and verify fragmentation
ping -c 1 -s 1472 -M do -f -W 3 192.168.1.1

The -M do flag sets the "Don't Fragment" bit. If packets are fragmented, the path does not support jumbo frames.

Also, verify that all routers have consistent MTU settings:

# On each router
ip link show dev eth0
ip link show dev eth1

If MTU is 1,500 on one link and 9,000 on another, and no jumbo frame support is configured, large packets are dropped or fragmented, increasing latency and reducing throughput.

Validating the Plan in a Test Environment

Before rolling out changes, test the entire network configuration under simulated load.

Use tc (traffic control) to emulate real-world conditions:

# Add 100ms delay, 2% loss, and 20% bandwidth limit on eth0
tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit
tc qdisc add dev eth0 parent 1:1 handle 10: netem delay 100ms loss 2% rate 100mbit

This emulates a 100ms latency, 2% packet loss, and 100 Mbps bandwidth on the interface.

Then run iperf3 under this simulated network:

iperf3 -c 192.168.1.10 -t 60 -i 10 -b 100M -l 131072 -P 8 -u -R

The -R flag reverses the direction, measuring download performance. Compare results against expected values.

Finally, validate that all components are correctly synchronized:

# Check all services
systemctl is-active --all | grep -E "(network|bgp|keepalived|iperf3)"
journalctl -u network --since "1 hour ago" --no-pager

This ensures that all network services are running, with no recent failures.

Iterative Planning and Review

Capacity planning is not a one-time task. Review the plan quarterly:

Update the configuration files, run tests, and document the changes.

Use git to track evolution:

git add .
git commit -m "Update network capacity plan: 208 Mbps peak, 12% growth rate"
git push origin main

This creates a versioned record of the network’s capacity planning process, enabling continuous improvement.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.