Computer Vision Pipelines

Design production CV systems for object detection, segmentation, OCR, and video analysis with preprocessing and post-processing.

A computer vision pipeline processes visual data through structured stages of preprocessing, model inference, post-processing, and output generation. This pattern applies when building systems that analyze images or video streams for classification, detection, segmentation, or feature extraction tasks. The pipeline's structure determines how input flows through processing stages, where each stage can be independently developed, tested, and deployed.

Data Ingestion and Preprocessing

Vision pipelines begin with reading and normalizing input data. Most systems accept image files in formats like JPEG, PNG, or TIFF via direct file paths or HTTP endpoints. The standard approach uses libraries such as OpenCV or PIL to load and convert images into tensors for model consumption.

Example preprocessing step:

import cv2
import numpy as np

def preprocess_image(image_path, target_size=(224, 224)):
    img = cv2.imread(image_path)
    img = cv2.resize(img, target_size)
    img = img.astype(np.float32) / 255.0
    return img

When dealing with video inputs, the pipeline must handle frame extraction and temporal consistency. Common failure mode: incorrect timestamp handling leading to misaligned frames in object tracking applications.

The pipeline expects consistent data types and dimensions across all input sources. For example, models trained on 224x224 RGB images will fail silently if given 1080p grayscale inputs without explicit conversion.

Model Inference Execution

Inference executes the core computer vision model on preprocessed inputs. The most common formats are ONNX or TensorFlow SavedModel for model serialization. The execution environment must match the model's requirements, including CPU/GPU allocation and memory constraints.

A basic inference call:

curl -X POST http://localhost:8080/vision/infer \
  -H "Content-Type: application/json" \
  -d '{"image": "base64_encoded_image", "model_id": "resnet50"}'

Model inference responses include bounding boxes, class probabilities, or segmentation masks depending on the task type. Incorrect model versioning causes silent failures where predictions match old model behavior despite new training.

Output Post-processing

Post-processing transforms raw model outputs into usable formats. This includes non-maximum suppression for detection tasks, mask merging for segmentation, or probability thresholding for classification. Most systems support configurable output formats including JSON, CSV, or binary representations.

Example post-processing for object detection:

def postprocess_detections(predictions, confidence_threshold=0.5):
    boxes = []
    for pred in predictions:
        if pred['confidence'] > confidence_threshold:
            boxes.append({
                'bbox': pred['bbox'],
                'class': pred['class'],
                'confidence': pred['confidence']
            })
    return boxes

Incorrect parameter values during post-processing often produce incorrect or empty outputs. For instance, setting a threshold too high can eliminate all valid detections.

Pipeline Configuration and Deployment

Pipelines are typically configured via environment variables or JSON files. Common keys include input paths, model identifiers, output formats, and processing parameters. The configuration must specify whether each stage is synchronous or asynchronous.

Example configuration:

{
  "input": {
    "type": "file",
    "path": "/data/images"
  },
  "model": {
    "id": "resnet50",
    "version": "v1.2"
  },
  "output": {
    "format": "json",
    "path": "/results"
  }
}

The pipeline must handle failures gracefully, especially during model loading or execution. A missing model file results in an error response rather than a silent failure.

Error Handling and Monitoring

Each stage in the pipeline should report clear error codes and messages. Common errors include:

Error responses follow consistent structure:

{
  "error": "ModelNotFound",
  "message": "Model with ID 'resnet50' not available in registry",
  "timestamp": "2023-01-01T12:00:00Z"
}

Monitoring focuses on throughput, latency, and error rates per stage. A pipeline that processes images at 100ms average but fails 5% of requests due to memory limits will appear healthy until a burst load reveals the underlying issue.

Batch vs Stream Processing

Pipelines support both batch and streaming inputs. Batch mode processes multiple images sequentially with fixed processing parameters, while streaming mode handles real-time data with dynamic configuration. The choice impacts resource allocation and response timing.

Streaming pipelines require buffering and rate limiting to handle variable input speeds. A common failure is unbounded buffer growth leading to memory exhaustion during high-volume periods.

Pipeline Scaling and Resource Management

Most vision pipelines scale by adding more processing units or increasing compute resources. The key is maintaining consistent input/output behavior across scaled instances. Resource constraints often manifest as timeouts or incomplete processing rather than model errors.

Example resource specification:

resources:
  cpu: "2"
  memory: "4Gi"
  gpu: "1"

Memory allocation directly affects batch size and processing speed. Under-provisioned GPUs cause frequent out-of-memory errors during inference.

Integration Points

Pipelines often integrate with external systems for input sourcing or result storage. Common integration points include:

Each integration point must be robust to network failures and data format changes. Misconfigured connections can cause silent data loss or incorrect processing.

Version Control and Rollbacks

Pipelines should version both code and model dependencies. The configuration file references specific model versions, ensuring reproducible results. Rollbacks require careful handling of model registry states and pipeline state.

The pipeline must support rollback to previous working configurations when new versions introduce errors. Version mismatches between models and inference code cause silent failures in production.

Performance Optimization

Optimization strategies include:

Each optimization introduces complexity that can obscure failure modes. For example, caching may mask model performance issues until cache invalidation occurs.

Failure Modes and Recovery

Silent failures are common in vision pipelines. A misconfigured preprocessing step can produce valid-looking outputs that are semantically incorrect. The pipeline should include explicit validation steps to catch such errors.

Recovery mechanisms include:

The most challenging failure is when a model update introduces a subtle change in output behavior that only becomes apparent after deployment.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.