Survival Analysis for Churn

Survival analysis models the time until an event occurs—here, customer churn. It is essential when users may leave at any point during their lifecycle, and when retention depends on duration, not just binary outcomes. This approach is indispensable when churn is not a fixed-rate phenomenon, when cohorts vary in behavior, when users have censored data (e.g., those still active at the end of the observation window), and when decisions depend on predicting *when* a customer will leave, not just *if* they will.

Estimating Churn Probability Over Time

The core task: estimate the probability that a user remains active past a given time, given their history. The survival function $ S(t) $ gives the probability that a user survives beyond time $ t $. In practice, this begins with a dataset of user events, where each row represents a user, and includes a start date and an optional end date (if the user churned) or a last activity timestamp (for right-censoring).

-- Input: users table with user_id, created_at, last_activity_at, and churned_at
-- If churned_at is NULL, user is right-censored at current date.

SELECT
  user_id,
  created_at,
  COALESCE(churned_at, CURRENT_DATE) AS event_time,
  EXTRACT(EPOCH FROM (COALESCE(churned_at, CURRENT_DATE) - created_at)) / 86400.0 AS days_since_creation,
  CASE
    WHEN churned_at IS NOT NULL THEN 1
    ELSE 0
  END AS censored
FROM users
WHERE created_at >= '2023-01-01';

The result is a dataset suitable for survival modeling. The two most common models—Kaplan-Meier and Cox proportional hazards—are used depending on whether the goal is descriptive or predictive.

Kaplan-Meier Estimator

Use Kaplan-Meier when the focus is on estimating survival curves without covariates. It is nonparametric, robust to censored data, and interpretable.

from lifelines import KaplanMeierFitter
import pandas as pd


df = pd.read_sql("""
  SELECT
    user_id,
    created_at,
    COALESCE(churned_at, CURRENT_DATE) AS event_time,
    CASE WHEN churned_at IS NOT NULL THEN 1 ELSE 0 END AS censored
  FROM users
  WHERE created_at >= '2023-01-01'
""", conn)

# Prepare for Kaplan-Meier: survival analysis with time-to-event and event indicator
kmf = KaplanMeierFitter()
kmf.fit(durations=df['event_time'] - df['created_at'], event_observed=df['censored'])

# Plot survival curve
kmf.plot_survival_function()

Key subtleties:

• The durations input must be in the same units (days, hours, etc.) across all users.
• event_observed is a boolean array: True if the user churned, False if censored.
• The model assumes that the hazard (churn rate) is piecewise constant between events.
• The survival curve is stepwise: it drops at each churn time, and remains flat until the next churn.

Failure mode: mixing up durations and event_times. A user who joined on 2023-01-01 and churned on 2023-03-01 has duration 60 days, but if event_time is used as the time-to-event, the model assumes 60 days from created_at, which is correct. But if created_at is not aligned with event_time, the durations are misestimated.

Modeling Time-to-Churn with Cox Proportional Hazards

When you want to predict churn based on user attributes—e.g., sign-up source, monthly active days, feature usage—use the Cox model. It estimates the hazard function $ h(t|X) = h_0(t) \cdot \exp(\beta^T X) $, where $ h_0(t) $ is the baseline hazard and $ X $ are covariates.

from lifelines import CoxPHFitter
import pandas as pd

# Add covariates: feature usage, plan type, etc.
df = pd.read_sql("""
  SELECT
    u.user_id,
    u.created_at,
    COALESCE(u.churned_at, CURRENT_DATE) AS event_time,
    CASE WHEN u.churned_at IS NOT NULL THEN 1 ELSE 0 END AS censored,
    COUNT(f.feature_id) AS total_features_used,
    AVG(f.usage_minutes) AS avg_minutes_per_day,
    u.plan_type
  FROM users u
  LEFT JOIN feature_usage f ON u.user_id = f.user_id
  WHERE u.created_at >= '2023-01-01'
  GROUP BY u.user_id, u.created_at, u.churned_at, u.plan_type
""", conn)

# Fit Cox model
cph = CoxPHFitter()
cph.fit(df, duration_col='event_time', event_col='censored', formula='plan_type + total_features_used + avg_minutes_per_day')

# View coefficients
cph.print_summary()

# Predict hazard ratio for a new user
new_user = pd.DataFrame({
  'plan_type': ['premium'],
  'total_features_used': [15],
  'avg_minutes_per_day': [45.0],
  'event_time': [pd.Timestamp('2023-06-01')],
  'censored': [False]
})

cph.predict_partial_hazard(new_user)

Critical details:

• formula uses a string expression similar to R’s formula syntax: var1 + var2 adds linear terms, log(var) applies logarithm, var1:var2 adds interaction.
• duration_col must be time-to-event, not absolute timestamps.
• event_col is the boolean indicator for event occurrence (churn).
• The model assumes proportional hazards: the hazard ratio between any two users remains constant over time.

Common failure modes:

• Covariates not time-invariant: if a user’s plan_type changes mid-lifecycle, but the model assumes it is fixed at time of entry, the hazard ratio becomes biased.
• Time-varying covariates: the Cox model supports strata and time_varying_covariates, but they require careful preprocessing. Each user must have a row per time interval where a covariate changes.

# Prepare time-varying data: one row per interval
time_varying_data = pd.read_sql("""
  SELECT
    user_id,
    start_time,
    end_time,
    plan_type,
    total_features_used,
    avg_minutes_per_day
  FROM time_varying_features
  WHERE start_time >= '2023-01-01'
""", conn)

cph = CoxPHFitter()
cph.fit(
  time_varying_data,
  id_col='user_id',
  event_col='censored',
  duration_col='end_time',
  start_col='start_time',
  formula='plan_type + total_features_used + avg_minutes_per_day'
)

Key pitfall: failing to specify id_col and start_col. Without them, the model treats each row as an independent user, not as intervals for a single user. This leads to underestimation of the number of events and inflated variance.

Predicting Individual Churn Times

Once a model is fitted, the next step is to predict individual churn times. This is used for proactive retention campaigns, personalized notifications, and churn risk scoring.

# Predict survival probability at specific times
survival_at_30 = cph.predict_survival_function(new_user, times=[30, 90, 180])

# Get median survival time
median_survival = cph.predict_median(new_user)

# Predict cumulative hazard
cumulative_hazard = cph.predict_cumulative_hazard(new_user, times=[30, 90, 180])

# Output: survival at 30, 90, 180 days
survival_at_30

The output is a DataFrame with one column per time point, and one row per user.

Interpreting Survival Curves

• Survival function $ S(t) $: probability user survives beyond $ t $ days.
• Hazard function $ h(t) $: instantaneous churn rate at time $ t $, estimated from the survival function.
• Cumulative hazard $ H(t) = -\log S(t) $: total risk accumulated by time $ t $.

Use predict_survival_function to get $ S(t) $, and predict_cumulative_hazard to get $ H(t) $. The two are inversely related: higher cumulative hazard implies lower survival.

Failure modes:

• Incorrect time units: using days in model but predicting for hours, or mixing days and months in times argument.
• Missing times argument: if times is not provided, predict_survival_function returns a function object, not a DataFrame. This causes errors when serializing predictions to JSON or storing in a database.

# Save predictions to database
predictions = cph.predict_survival_function(new_user, times=[30, 60, 90, 180])
predictions.reset_index(inplace=True)
predictions = predictions.rename(columns={'index': 'user_id'})
predictions.to_sql('churn_predictions', conn, if_exists='replace', index=False)

Validating and Monitoring Models in Production

Churn models are not one-off analyses. They must be retrained and monitored.

Retraining Pipeline

# Run daily: fetch latest user data, retrain model, export predictions
#!/bin/bash

# 1. Export current user data
psql -c "
  COPY (
    SELECT
      user_id,
      created_at,
      COALESCE(churned_at, CURRENT_DATE) AS event_time,
      CASE WHEN churned_at IS NOT NULL THEN 1 ELSE 0 END AS censored,
      COUNT(f.feature_id) AS total_features_used,
      AVG(f.usage_minutes) AS avg_minutes_per_day,
      u.plan_type
    FROM users u
    LEFT JOIN feature_usage f ON u.user_id = f.user_id
    WHERE created_at >= (CURRENT_DATE - INTERVAL '30 days')
    GROUP BY u.user_id, u.created_at, u.churned_at, u.plan_type
  ) TO STDOUT WITH CSV HEADER
" -d analytics > /tmp/users_for_model.csv

# 2. Train model
python train_churn_model.py /tmp/users_for_model.csv

# 3. Save model to disk
mv model.pkl /models/cph_churn_$(date +%Y-%m-%d).pkl

# 4. Deploy to API
systemctl restart churn-predictor-api

Model Monitoring

Monitor key metrics:

• Number of users in cohort: ensures data is being ingested.
• Number of churn events per day: detects data pipeline lags.
• Median survival time: tracks changes in overall retention.
• C-index (concordance index): measures model’s ability to rank users by churn time.

# Compute C-index from test data
cph = CoxPHFitter()
cph.fit(test_data, duration_col='event_time', event_col='censored')
c_index = cph.concordance_index_

# C-index: 0.5 = random, 1.0 = perfect
print(f"C-index: {c_index:.4f}")

Failure modes:

• C-index computed on training data: the model is overfitted, so C-index on training data is inflated. Always compute it on held-out test data.
• C-index with time-varying covariates: the C-index assumes that the predicted hazard is evaluated at a fixed time, but for time-varying models, it must account for time-dependent risk.

Error Patterns

• Missing event_col: if censored is NULL, the Cox model treats NULL as False, but TRUE is expected. This leads to underestimation of churn events.
• Incorrect duration_col: if event_time is a timestamp but duration_col is a INTERVAL, the model fails to align time scales.
• Covariates with missing values: CoxPHFitter drops rows with NaN in any covariate, but the user expects imputation. Always preprocess missing data.
• Model not retrained daily: churn predictions lag by days, leading to late retention actions.

Handling Censoring and Time-Dependent Events

Censoring is the hallmark of survival analysis, but it is often misunderstood.

• Right-censoring: user is active at end of observation; churn time is unknown.
• Left-censoring: user joined before observation window; event time is unknown.
• Interval-censoring: event occurred between two known times.

In practice, only right-censoring is common. Left-censoring is rare, but critical when analyzing cohorts with historical data.

-- Example: users who signed up before 2023-01-01, observed from 2023-01-01 to 2023-12-31
SELECT
  user_id,
  created_at,
  '2023-01-01' AS start_time,
  '2023-12-31' AS end_time,
  COALESCE(churned_at, '2023-12-31') AS event_time,
  CASE WHEN churned_at IS NOT NULL THEN 1 ELSE 0 END AS censored
FROM users
WHERE created_at < '2023-01-01';

Use start_col in Cox model for interval-censored data.

When events are not single-timepoints but sequences—e.g., users can churn multiple times, or churn is marked by multiple actions—use recurrent event models.

from lifelines import RecurrentEventFitter

# Input: one row per event per user
events = pd.read_sql("""
  SELECT
    user_id,
    event_time,
    event_type
  FROM churn_events
  WHERE event_type = 'churn'
""", conn)

ref = RecurrentEventFitter()
ref.fit(events, id_col='user_id', event_col='event_time')

# Predict next churn time
next_churn = ref.predict_survival_function(events, times=[30, 90])

Recurrent models handle the fact that the hazard of churn depends on previous events. They are essential when churn is not a single event but a process.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.