Vision API Pipelines

Vision API Pipelines describe the sequence of automated steps that take raw visual input—images or video frames—through preprocessing, model inference, and result handling. They matter whenever you need to extract structured information from visual data without writing custom image‑processing code for every new format, destination, or model version. This page covers the practical steps of wiring such a pipeline with off‑the‑shelf tools, the concrete commands you can run today, and the failure modes that catch teams off guard after they’ve been running in production for a few weeks.

Prerequisites & Environment

Before assembling a vision pipeline, ensure the following are available on the machine or cluster where the pipeline will run:

• Docker (or a container runtime compatible with docker) for isolating each pipeline step. Version 20.10+ is recommended for consistent flag behavior.
• curl for ad‑hoc HTTP calls to inference endpoints. No special build flags are needed; the default install provides TLS and proxy support.
• kubectl if the pipeline runs inside a Kubernetes cluster. Version must match the cluster’s API server version to avoid Unable to connect to the server errors.
• psql (or any PostgreSQL‑compatible CLI) for persisting pipeline metadata, run logs, and result tables. Default port 5432; adjust with -h if needed.
• A model serving endpoint that accepts image data via HTTP POST and returns JSON. The endpoint may be a TensorFlow Serving container, an ONNX Runtime gRPC/HTTP gateway, or any custom service conforming to the interface described below. No specific product’s internal config is assumed.

A minimal working directory layout might look like:

vision-pipeline/
├── Dockerfile
├── entrypoint.sh
├── frames/          # input images or video frames
└── results/         # output directory for JSON results

Wiring a Minimal Pipeline

The fastest way to get a vision pipeline running is to chain three Docker containers: one that resizes and normalizes frames, one that sends them to an inference endpoint, and one that writes the output to disk or a database.

Step 1 – Preprocess frames A simple entrypoint.sh can use ImageMagick (available in most base images) to resize and convert color space:

#!/bin/bash
set -euo pipefail


convert "$1" -resize 224x224 -colorspace RGB /data/out/normalized.jpg

Build a tiny image:

docker build -t vision-preprocess -f Dockerfile.preprocess .

Run it on a frame:

docker run --rm -v $(pwd)/frames:/data frames/input.jpg
# normalized.jpg now exists at /data/out/normalized.jpg on the host

Step 2 – Invoke the inference endpoint Assume the model server listens on http://localhost:8080/v1/infer and expects a JSON payload with a base64‑encoded image under the key input. A single curl command can dispatch the preprocessed frame:

curl -s -X POST http://localhost:8080/v1/infer \
  -H "Content-Type: application/json" \
  -d "{\"input\":\"$(base64 -w0 /data/out/normalized.jpg)\"}" \
  -o /data/out/inference.json

Flags broken down:

• -s silences progress output.
• -X POST selects the HTTP method.
• -H "Content-Type: application/json" tells the server the payload format.
• -d supplies the JSON body; base64 -w0 encodes the file without wrapping lines.
• -o redirects stdout to a local file.

If the server is not reachable, curl returns:

curl: (7) Failed to connect to localhost port 8080: Connection refused

or, if the server closes the connection prematurely:

curl: (52) Empty reply from server

Step 3 – Persist or display results The JSON shape returned by most vision inference services includes at minimum label, confidence, and bounding_box. A quick psql insert can log the run:

psql -h db-host -U user -d visiondb -c \
  "INSERT INTO pipeline_runs (run_id, frame_path, label, confidence, received_at) \
   VALUES ('$(date +%s)', '/data/frames/input.jpg', '$((jq -r .label /data/out/inference.json)', \
   $((jq -r .confidence /data/out/inference.json)', now());"

If jq is not installed, install it via the system package manager (apt-get install jq, brew install jq) before running the command.

Invoking an Inference Endpoint at Scale

When throughput exceeds a single curl call, replace the manual loop with a lightweight script or a container orchestrated by kubectl. Below is a bash loop that processes all frames in a directory and respects a per‑request timeout:

#!/bin/bash
set -euo pipefail
TOKEN="${VISION_API_TOKEN:-}"
ENDPOINT="${VISION_API_ENDPOINT:-http://localhost:8080/v1/infer}"

for frame in frames/*.jpg; do
  payload="{\"input\":\"$(base64 -w0 "$frame")\"}"
  result=$(curl -s -m 15 -X POST "$ENDPOINT" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $TOKEN" \
    -d "$payload" -o "/results/$(basename "$frame".json)")
  if [ $? -ne 0 ]; then
    echo "FAILED: $frame at $(date --iso-8601=seconds)" >> /logs/errors.log
  fi
done

Key flags and their roles:

• -m 15 sets a 15‑second timeout per request. Without it, a hung model server would block the loop indefinitely.
• -H "Authorization: Bearer $TOKEN" injects auth if the endpoint requires it. Omit the flag entirely for unauthenticated services; adding an empty or malformed token is a common source of 401 Unauthorized errors.
• The loop writes per‑frame JSON to /results/. If the directory lacks write permission, the shell exits with Permission denied and no results are persisted.

Deploying the same logic under Kubernetes reduces manual boilerplate. A minimal pipeline-job.yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: vision-pipeline-job
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: pipeline
        image: alpine:3.19
        command: ["sh", "-c"]
        args:
          - |
            apk add --no-cache curl jq
            for frame in /frames/*.jpg; do
              payload="$(jq -n --arg img "$(base64 "$frame")" '{input: $img}')"
              curl -s -m 15 -X POST http://model-svc:8080/v1/infer \
                -H "Content-Type: application/json" \
                -d "$payload" -o "/output/$(basename "$frame".json)"
            done
        volumeMounts:
        - name: frames
          mountPath: /frames
        - name: output
          mountPath: /output
      volumes:
      - name: frames
        persistentVolumeClaim:
          claimName: frames-pvc
      - name: output
        persistentVolumeClaim:
          claimName: results-pvc

Apply with:

kubectl apply -f pipeline-job.yaml
kubectl logs job/vision-pipeline-job

Post‑Processing & Storing Results

Inference output is rarely ready for downstream consumption without a post‑processing step. Common operations include:

• Non‑maximum suppression (NMS) to merge overlapping bounding boxes from multi‑scale evaluation.
• Confidence thresholding to discard detections below a user‑defined percentile.
• Label mapping from model‑internal IDs to human‑readable tags stored in a lookup table.

A compact Python snippet that reads the JSON produced by the pipeline and writes a CSV suitable for BI tools:

#!/usr/bin/env python3
import json, csv, sys, pathlib

inp = pathlib.Path(sys.argv[1]).read_text()
data = json.loads(inp)

records = []
for det in data.get("detections", []):
    if det["confidence"] < 0.5:
        continue
    records.append({
        "frame": pathlib.Path(sys.argv[1]).stem,
        "label": det["label"],
        "score": det["confidence"],
        "xmin": det["bbox"][0],
        "ymin": det["bbox"][1],
        "xmax": det["bbox"][2],
        "ymax": det["bbox"][3],
    })

out = pathlib.Path(sys.argv[2])
out.parent.mkdir(parents=True, exist_ok=True)
with out.open("w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["frame","label","score","xmin","ymin","xmax","ymax"])
    writer.writeheader()
    writer.writerows(records)

Run it as:

./postprocess.py /data/out/inference.json /results/output.csv

If the input JSON lacks a detections key, the script silently produces an empty CSV. This “silent empty output” pattern is a frequent source of downstream pipeline failures when teams assume records exist.

Sharp Edges – What Silently Fails

Symptom	Root Cause	What to Check
All results show “person” with 1.0 confidence	Color space mismatch: the pipeline resized frames in BGR (OpenCV default) while the model was trained on RGB data.	Verify the preprocessing step explicitly converts colorspace (-colorspace RGB for ImageMagick, cv2.cvtColor(img, cv2.COLOR_BGR2RGB) for OpenCV).
504 Gateway Timeout on every request	Model server enforces a strict input tensor shape (e.g., 1x3x224x224) but the pipeline sends variable‑size images without central cropping or padding.	Check the server’s accepted input shape in its OpenAPI/Swagger spec or documentation. Add a resizing/cropping step before the POST.
Results appear duplicated in the database	The pipeline loop does not flush or close the HTTP connection, and the server treats each curl as a separate session, writing a new row per connection.	Ensure each iteration creates a fresh curl invocation (the script above already does this). If using a persistent client, call .close() or use a connection pool with explicit session IDs.
403 Forbidden on kubectl‑deployed pipeline	The job’s service account lacks permission to write to the results PVC, or the PVC is bound in read‑only mode.	Describe the PVC: kubectl describe pvc results-pvc. Confirm the service account has create, update rights, or bind a different volume.
Inference JSON contains null for bounding_box	The model returned a “no‑detect” score below the internal threshold, but the pipeline did not filter on confidence before parsing.	Add a guard: if det.get("bounding_box") is None: continue before accessing array indices.

Production‑learned lessons

• Always log the raw request payload and raw response body for any frame that results in a non‑2xx HTTP code. The cost of storage is negligible compared with the time spent debugging a mismatched tensor shape.
• Version‑pin the Docker image used for each pipeline step. A docker tag change without a corresponding docker pull on all nodes leads to “image not found” errors that look like network failures.
• If the inference endpoint supports batching (accepting an array of images in one POST), measure the latency trade‑off. Batching often reduces per‑frame overhead but increases memory pressure on the model server; a sudden OOM kill will manifest as Kill signal 9 in the container logs without a clean exit code.
• When piping image data through base64 encoding, payloads exceeding a few megabytes can trigger proxy timeouts or Kubernetes Limit-Request-Size rejections. For large video frames, consider writing the encoded string to a temporary file and mounting it into the curl command via -T instead of embedding it in JSON.

Extending the Pipeline

Batch processing If the model server exposes a /v1/batch-infer endpoint that accepts {"inputs": [base64_str1, base64_str2, ...]}, the curl command adjusts to:

curl -s -X POST http://localhost:8080/v1/batch-infer \
  -H "Content-Type: application/json" \
  -d "$(jq -n --argjson imgs "$(cat frames/*.jpg | xargs base64 -w0 | jq -R .)" '{inputs: $imgs}')" \
  -o /results/batch.json

The --argjson flag ingests the output of base64 as a JSON array. If the server does not support batching, the loop per‑frame remains the safe fallback.

Async coordination For pipelines where inference latency varies significantly (e.g., edge devices with intermittent connectivity), use a message queue to decouple frame submission from result collection. A minimal RabbitMQ setup:

# Publish a frame for processing
printf '{"frame":"%s","path":"%s"}' "$(basename "$frame")" "$frame" | \
  rabbitmqadmin publish exchange=vision routing_key=infer body=@-

# Consume results (separate process)
rabbitmqadmin get queue=results acknowledge=true

Acknowledge each message only after the downstream post‑processing step completes successfully, preventing message loss if the consumer crashes mid‑processing.

Observability Add a Prometheus exporter sidecar to the pipeline container that increments a counter vision_pipeline_requests_total with labels status="success" or status="error", and a histogram vision_inference_latency_seconds. The exporter listens on :9119/metrics; scrape it with a Prometheus server to visualize trends and detect sudden latency spikes that often precede model degradation.

Summary

Vision API pipelines are a practical way to turn raw visual input into structured data without writing bespoke image‑processing code for every new use case. The concrete steps—preprocess, POST to an inference endpoint, handle the JSON response, and persist results—can be assembled today with docker, curl, jq, and a small amount of shell or Python glue. The sharp edges listed above are not academic; they are the exact failure modes that appear in logs when a pipeline moves from a developer’s laptop to a production cluster. Keep the preprocessing color‑space explicit, version‑pin your Docker images, log raw payloads and responses, and treat each pipeline step as an independently scalable unit. That approach keeps the pipeline reliable as volume, latency, and model complexity grow.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.