Statistical Process Control

Production engineering guide for statistical process control covering patterns, implementation strategies, and operational best practices.

Statistical Process Control (SPC) is the practice of monitoring, controlling, and improving a process using statistical methods. It matters when a system generates continuous data—sensor readings, transaction logs, service latencies, or manufacturing outputs—and the goal is to detect deviations from expected behavior *before* they impact users, customers, or product quality. SPC is not about one-off experiments or isolated metrics; it is about *ongoing* process health, where control charts, process capability indices, and real-time alerts form the backbone of operational stability.

Monitoring a Process Over Time

The primary objective is to observe a process’s output over time and detect shifts or drifts in its behavior. The fundamental tool is the control chart, typically a line plot of a key metric—e.g., mean latency, error rate, throughput—over time, with upper and lower control limits (UCL and LCL) and a centerline (CL) representing the process mean.

The process begins with collecting samples at regular intervals. For a service logging response times every minute, the data is a sequence of timestamps and measured values:

[
  { "timestamp": "2024-03-15T08:00:00Z", "latency_ms": 142 },
  { "timestamp": "2024-03-15T08:01:00Z", "latency_ms": 138 },
  { "timestamp": "2024-03-15T08:02:00Z", "latency_ms": 151 },
  ...
]

Each point is plotted on a chart. The centerline is the average of the first 20–30 data points, usually computed offline and stored as a metadata value. The control limits are set at ±3 standard deviations from the mean, assuming normality. A point outside these limits signals a special cause variation—something unusual has occurred.

A common failure: using a single global mean as the centerline, but the process is seasonal or shifts over time. The solution is to use moving averages or weighted averages for the centerline, updating it in real time.

Computing Control Limits in Practice

Control limits are not just theoretical. They must be derived from actual process data, and the method affects sensitivity and false alarm rates.

To compute control limits dynamically, use the average moving range (AMR) for small sample sizes (e.g., n = 2 or 3 per subgroup). The AMR is the average of the absolute differences between consecutive data points.

Given a time series of individual values:

import numpy as np

def compute_control_limits(data, k=3):
    # Compute centerline as mean
    cl = np.mean(data)

    # Compute average moving range
    diffs = np.diff(data)
    amr = np.mean(np.abs(diffs))

    # Estimate standard deviation from AMR
    sigma = amr / 1.128  # 1.128 is the d2 constant for n=2

    # Compute control limits
    ucl = cl + k * sigma
    lcl = cl - k * sigma

    return {
        "centerline": cl,
        "upper_control_limit": ucl,
        "lower_control_limit": lcl,
        "sigma": sigma
    }

This function returns a dictionary that becomes the basis for the control chart. The key insight: k = 3 is standard, but k = 2.66 is common for individuals charts (I-charts), where the control limits are based on the moving range.

A silent failure: assuming the data is independent when it is not. Autocorrelation in time series—e.g., response times from a cached service—leads to underestimated control limits and too many false alarms. The remedy is to apply Shewhart’s correction by adjusting the control limits based on the autocorrelation coefficient, or using exponentially weighted moving averages (EWMA).

Detecting Process Shifts and Anomalies

Once a control chart is live, the next task is to detect shifts in the process: sudden changes in mean (e.g., a new deployment), increases in variability (e.g., a network outage), or trends (e.g., gradual degradation).

Shift Detection: Run Rules

A single point outside control limits is not enough. Run rules—simple, rule-based tests—add sensitivity.

Common run rules:

These are checked in real time. For example, a service logs a new latency value every minute. The system maintains a buffer of the last 10 values. Every minute, it checks:

def evaluate_run_rules(data):
    n = len(data)
    cl = np.mean(data[:n-1])  # previous mean
    ucl = cl + 3 * np.std(data[:n-1])
    lcl = cl - 3 * np.std(data[:n-1])

    # Rule 1: One point beyond 3σ
    if data[-1] > ucl or data[-1] < lcl:
        return {"rule_1": True, "signal": "special cause"}

    # Rule 2: Nine in a row on same side
    last_nine = data[-9:]
    if (np.all(last_nine >= cl) or np.all(last_nine <= cl)):
        return {"rule_2": True, "signal": "shift in mean"}

    # Rule 3: Six in a row increasing or decreasing
    if (np.all(np.diff(last_nine) > 0)) or (np.all(np.diff(last_nine) < 0)):
        return {"rule_3": True, "signal": "trend"}

    # Rule 4: Four out of five in outer 2σ
    outer_2sigma = (data[-5:] > (cl + 2 * np.std(data[:n-1]))) | (data[-5:] < (cl - 2 * np.std(data[:n-1])))
    if np.sum(outer_2sigma) >= 4:
        return {"rule_4": True, "signal": "increased variability"}

    return {"signal": "no rule triggered"}

A critical edge: the choice of “outer 2σ” region is not standardized. Some use 2σ as a threshold, others use the distance from centerline to 2σ as the region. The difference matters when the process is non-normal. A rule that works well for a Gaussian process may fail for heavy-tailed data.

Process Capability and Performance

Beyond monitoring, SPC aims to assess whether the process is capable—able to produce output within specification limits (LSL and USL).

Suppose a service must respond under 200ms (USL) and at least 100ms (LSL). The process capability index Cp measures how well the process fits within these limits relative to its natural variation:

Cp = (USL - LSL) / (6 * σ)

If Cp = 1, the process uses 6σ of its total variation to meet requirements. Cp > 1.33 is considered capable.

But Cp assumes the process is centered. Cpk accounts for centering:

Cpk = min( (USL - mean) / (3 * σ), (mean - LSL) / (3 * σ) )

Cpk < Cp indicates the process is off-center.

To compute Cpk in practice, use a SQL query over a historical dataset:

WITH stats AS (
  SELECT
    AVG(latency_ms) AS mean,
    STDDEV(latency_ms) AS sigma
  FROM service_logs
  WHERE timestamp >= '2024-02-01T00:00:00Z'
)
SELECT
  (200 - 100) / (6 * sigma) AS Cp,
  LEAST(
    (200 - mean) / (3 * sigma),
    (mean - 100) / (3 * sigma)
  ) AS Cpk
FROM stats;

A silent failure: using the sample standard deviation (n-1) instead of population standard deviation (n). For capability analysis, the process is assumed to be stable and long-term, so using σ = √(Σ(xi - μ)² / n) is preferred. The wrong choice inflates Cp and underestimates Cpk.

Another edge: Cpk is sensitive to outliers. A single 500ms spike can pull the mean and increase σ, reducing Cpk. To counter this, use median absolute deviation (MAD) to estimate robust σ:

sigma_mad = 1.4826 * np.median(np.abs(data - np.median(data)))

This robust σ is less affected by outliers and gives a more accurate picture of long-term process capability.

Real-Time Alerts and Maintenance

SPC is not passive. It must trigger actions when a process deviates.

Alerts are sent when:

These alerts are typically delivered via HTTP or message queues. A common pattern uses a POST /alerts endpoint:

{
  "timestamp": "2024-03-15T08:15:00Z",
  "process": "api-gateway-latency",
  "metric": "latency_ms",
  "value": 215,
  "control_limits": {
    "centerline": 142,
    "ucl": 186,
    "lcl": 98
  },
  "rule_triggered": "rule_2",
  "c_pk": 1.04,
  "severity": "warning"
}

The alert system listens for these events and routes them to appropriate teams, systems, or incident tools.

A sharp edge: the timing of alert triggers. A rule is evaluated every minute, but the alert is not sent immediately. The system may batch alerts and send them every 5 minutes. This introduces latency—a 200ms spike at 08:00 is detected at 08:05, but the alert is sent at 08:07. This can mask short-lived issues.

To fix this, use event-driven alerting: each rule evaluation triggers a new alert event, which is stored in a Kafka stream or message queue. Consumers pull events in real time and trigger actions—e.g., a Slack notification, a PagerDuty incident, or a rollout of a new service configuration.

A failure mode: alert fatigue. Too many alerts, too many rules, and teams start ignoring them. The solution is alert deduplication based on time windows and metric combinations.

For example, if the same process shows a Rule 1 and Rule 2 alert within 10 minutes, the system combines them into a single incident. This reduces noise and ensures that only significant, sustained deviations trigger action.

Maintaining the Control Chart Over Time

A control chart is not static. Over time, the process changes—new hardware, code deployments, increased load. The control limits must be updated.

The standard approach: recalibrate the control chart every 25–50 data points (or every 1–2 weeks). But this requires a mechanism to store and version the chart’s parameters.

Each control chart is defined by:

These are stored as configuration in a database or a version-controlled file.

Example configuration in JSON:

{
  "chart_id": "api-latency-1m",
  "metric": "latency_ms",
  "source": "service_logs",
  "sampling_interval": "1m",
  "chart_type": "individuals",
  "centerline_method": "moving_average",
  "control_limits": {
    "method": "amr",
    "k": 3,
    "update_interval": "1d"
  },
  "run_rules": [
    { "name": "rule_1", "condition": "point > 3σ" },
    { "name": "rule_2", "condition": "9_points_same_side" },
    { "name": "rule_3", "condition": "6_points_monotonic" }
  ],
  "recalibration_interval": "7d",
  "alert_thresholds": {
    "c_pk": { "warning": 1.0, "critical": 0.8 },
    "run_rule": { "warning": 2, "critical": 5 }
  }
}

This config drives the entire SPC pipeline: data ingestion, chart computation, alerting, and reporting.

A subtle failure: using different data for control limits and for Cpk. The control chart uses a rolling window of recent data, while Cpk uses a fixed historical window. If the process changes, the control limits may be out of sync with the capability index.

The fix: align the data sources. Use the same time window for both, or use rolling Cpk—compute Cpk over a sliding window of 30 days, and compare it to the control limits computed over the same window.

Summary

Statistical Process Control is not a one-off analysis but a continuous feedback loop: measure → chart → detect → alert → act → recalibrate. It is the foundation of operational stability in data-intensive systems. Its power lies in its simplicity—control charts, run rules, and capability indices—combined with sharp, executable details: how to compute limits, when to trigger alerts, how to handle shifts, and how to maintain long-term process health. The tools are simple, the patterns are repeatable, and the failure modes are predictable. When a system is under SPC, it is not just monitored—it is *in control*.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.