Clustering Algorithms

Production engineering guide for clustering algorithms covering patterns, implementation strategies, and operational best practices.

Clustering algorithms group data points into meaningful subsets based on similarity, enabling discovery, segmentation, and dimensionality reduction. They are essential when you have unlabeled data and need to infer structure—such as customer segments from transaction logs, document topics from a corpus, or anomaly clusters in system telemetry—without predefined categories or training labels.

Choosing a Clustering Algorithm

Selecting a clustering algorithm depends on data size, dimensionality, cluster shape, and computational budget. Use k-means for high-volume, low-dimensional data with spherical clusters; DBSCAN for irregular shapes and noise resistance; hierarchical clustering for nested structures and interpretability; and Gaussian Mixture Models for probabilistic cluster assignments and soft boundaries.

k-means

Use k-means when you need fast, scalable clustering with a known number of clusters and reasonably spherical, compact groups.

Run k-means with the scikit-learn library in Python:

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import numpy as np


X = np.load("features.npy")  # Shape: (n_samples, n_features)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Fit k-means
kmeans = KMeans(n_clusters=5, init="k-means++", max_iter=300, random_state=42)
labels = kmeans.fit_predict(X_scaled)

# Save model and predictions
np.save("cluster_labels.npy", labels)

Key configuration options:

Failure modes:

Use the elbow method to determine n_clusters:

import matplotlib.pyplot as plt
from sklearn.metrics import silhouette_score

inertias = []
silhouettes = []
k_range = range(1, 21)

for k in k_range:
    kmeans = KMeans(n_clusters=k, init="k-means++", max_iter=300, random_state=42)
    kmeans.fit(X_scaled)
    inertias.append(kmeans.inertia_)
    silhouettes.append(silhouette_score(X_scaled, kmeans.labels_))

plt.plot(k_range, inertias, "bo-")
plt.xlabel("Number of clusters (k)")
plt.ylabel("Inertia")
plt.title("Elbow Method for k-means")
plt.savefig("elbow.png")

Silhouette score above 0.5 indicates good clustering; below 0.3 suggests overlapping or poorly separated clusters.

DBSCAN

Use DBSCAN when clusters vary in shape, density, and number of points, and when noise (outliers) must be detected automatically.

DBSCAN requires two parameters: eps (maximum distance between points in a cluster) and min_samples (minimum points to form a core cluster).

from sklearn.cluster import DBSCAN
import numpy as np

X = np.load("features.npy")
dbscan = DBSCAN(eps=0.3, min_samples=5, metric="euclidean")

labels = dbscan.fit_predict(X)

# Labels: -1 indicates noise points
np.save("dbscan_labels.npy", labels)

# Count clusters and noise
unique_labels, counts = np.unique(labels, return_counts=True)
n_clusters = len(unique_labels) - (1 if -1 in unique_labels else 0)
n_noise = counts[unique_labels == -1][0] if -1 in unique_labels else 0

Key options:

Failure modes:

To tune eps and min_samples, use the k-distance graph:

from sklearn.neighbors import NearestNeighbors
import matplotlib.pyplot as plt

# Compute k-nearest neighbors
knn = NearestNeighbors(n_neighbors=5).fit(X)
distances, indices = knn.kneighbors(X)

# Sort distances to each point
k_distances = np.sort(distances[:, -1])

# Plot k-distance graph
plt.plot(k_distances)
plt.xlabel("Point index (sorted by k-distance)")
plt.ylabel("k-distance")
plt.title("k-Distance Graph for DBSCAN")
plt.savefig("k-distance.png")

Look for a "knee" or "elbow" in the plot: the eps value at the knee point is a good candidate.

Hierarchical Clustering

Use hierarchical clustering when you want to explore cluster structure at multiple granularities or when the number of clusters is unknown.

from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import adjusted_rand_score
import numpy as np

X = np.load("features.npy")

# Perform hierarchical clustering
clustering = AgglomerativeClustering(
    n_clusters=10,
    linkage="ward",  # or "complete", "average", "single"
    distance_threshold=None
)

labels = clustering.fit_predict(X)

np.save("hierarchical_labels.npy", labels)

Key options:

To visualize the dendrogram, use scipy.cluster.hierarchy:

from scipy.cluster.hierarchy import dendrogram, linkage
import matplotlib.pyplot as plt

Z = linkage(X, method="ward")

plt.figure(figsize=(10, 6))
dendrogram(Z)
plt.title("Hierarchical Clustering Dendrogram")
plt.xlabel("Sample index")
plt.ylabel("Distance")
plt.savefig("dendrogram.png")

Failure modes:

Gaussian Mixture Models (GMM)

Use GMM when you need probabilistic cluster assignments, soft clustering, and clusters with different shapes and variances.

from sklearn.mixture import GaussianMixture
import numpy as np

X = np.load("features.npy")

gmm = GaussianMixture(
    n_components=4,
    covariance_type="full",  # or "tied", "diag", "spherical"
    init_params="kmeans",  # or "random"
    max_iter=200,
    random_state=42
)

labels = gmm.fit_predict(X)
probabilities = gmm.predict_proba(X)

np.save("gmm_labels.npy", labels)
np.save("gmm_probabilities.npy", probabilities)

Key options:

Failure modes:

Evaluate GMM using the Bayesian Information Criterion (BIC):

import numpy as np

bic_scores = []
n_components_range = range(1, 11)

for n in n_components_range:
    gmm = GaussianMixture(n_components=n, covariance_type="full", max_iter=200)
    gmm.fit(X)
    bic_scores.append(gmm.bic(X))

plt.plot(n_components_range, bic_scores, "ro-")
plt.xlabel("Number of components")
plt.ylabel("BIC")
plt.title("Model Selection for GMM")
plt.savefig("gmm_bic.png")

Choose the number of components at the minimum BIC value.

Evaluating Clustering Quality

Use internal and external metrics to assess cluster quality.

Internal Metrics

from sklearn.metrics import silhouette_score, calinski_harabasz_score

inertia = kmeans.inertia_  # From k-means fit
silhouette_avg = silhouette_score(X_scaled, labels)
calinski_harabasz = calinski_harabasz_score(X_scaled, labels)

External Metrics

When ground truth labels exist (e.g., from expert annotation), use:

from sklearn.metrics import adjusted_rand_score

ari = adjusted_rand_score(true_labels, predicted_labels)
from sklearn.metrics import normalized_mutual_info_score

nmi = normalized_mutual_info_score(true_labels, predicted_labels)

Common Failure Modes

Running Clustering at Scale

For large datasets, use distributed clustering tools like dask-ml or Spark MLlib.

from dask_ml.cluster import KMeans as DaskKMeans
from dask.distributed import Client

client = Client("scheduler-address:8786")

# Load data in parallel
X = client.persist(dask.dataframe.read_parquet("data/"))

# Fit k-means
kmeans = DaskKMeans(n_clusters=10, init="k-means++", max_iter=100)
kmeans.fit(X)

labels = kmeans.predict(X)

Use pandas or pyarrow for data input, joblib or cloudpickle for model serialization.

Common error messages:

Choosing the Right Algorithm: A Decision Flow

  1. Is the number of clusters known?
  1. Are clusters expected to be spherical?
  1. Is noise (outliers) important?
  1. Is data high-dimensional?
  1. Is interpretability of cluster hierarchy required?
  1. Are probabilistic assignments needed?

This flow ensures that clustering is not just an algorithmic step, but a structured, repeatable process in data pipelines.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.