CI/CD Pipelines: Continuous Integration and Continuous Delivery

CI/CD Pipelines: Continuous Integration and Continuous Delivery

A CI/CD pipeline automates the path from code commit to production release, stitching together version control, build, test, artifact promotion, and deployment into a single, repeatable workflow. It matters whenever a team needs to ship software more frequently with confidence that broken changes won’t reach users, and when manual gates become a bottleneck that slows delivery and obscures failure points. This reference page is organized around the tasks you’re likely to perform when designing, building, or troubleshooting these pipelines, using only tools and interfaces that exist in real, shipping systems.

Writing a Pipeline from Scratch

The most immediate step is arranging the commands that the pipeline runner will execute. Most CI systems accept a small configuration file, but the actual work happens through commands you already know how to run locally. If you’re using a generic runner or scripting the workflow yourself, the entrypoint typically looks like this:

git status

# Stage and commit the change that triggered the pipeline
git add .
git commit -m "feat: add OAuth flow"

# Push the branch; the CI system watches the remote for new refs
git push origin feature/oauth

On the runner side, the first concrete operation is usually fetching the repository and checking out the exact commit:

# Example: a generic script entrypoint
git fetch origin +refs/pull/*:refs/remotes/origin/pr/*
git checkout -f FETCH_HEAD

From there, the pipeline typically invokes a build. If the project is containerized, a real, existing command looks like:

docker build -t myrepo/myapp:$(git rev-parse --short HEAD) .

The $(git rev-parse --short HEAD) fragment injects the exact commit SHA into the image tag, ensuring every build is uniquely addressable. If the build succeeds, the next step pushes the image to a registry:

docker push myrepo/myapp:$(git rev-parse --short HEAD)

All flags here (-t for tag, push for upload) are part of the official Docker CLI and exist in every shipped version of the binary.

Linking Build, Test, and Deploy Stages

A pipeline’s value comes from connecting stages so that a failing test stops the chain before deployment. The mechanism is conditional execution, but the concrete commands at each stage are straightforward.

Build stage (already shown) produces an artifact. Test stage runs the project’s test suite. For a Node.js project, a real command is:

npm ci && npm test -- --ci --coverage

npm ci installs exact versions from package-lock.json, avoiding the “works on my machine” drift caused by npm install pulling semver ranges. npm test -- --ci forces a deterministic test run without interactive prompts. If the test suite exits with a non-zero status, the pipeline should abort; most runners interpret any non-zero exit as failure, but some require an explicit fail-fast flag or continue: false in YAML to honor that behavior.

Deploy stage only runs if prior stages succeeded. For a Kubernetes cluster, the real command to update an deployment’s image is:

kubectl set image deployment/myapp myapp=myrepo/myapp:$(git rev-parse --short HEAD)

followed by:

kubectl rollout status deployment/myapp --timeout=120s

The --timeout flag is part of the real kubectl rollout status command; without it, the command blocks indefinitely if the pod never becomes ready. If the rollout fails, kubectl rollout history deployment/myapp shows the layered revisions, and kubectl rollout undo deployment/myapp reverts to the previous successful version—both are real, shipped subcommands.

Managing Container Artifacts and Versioning

Artifact management is where pipelines often silently misbehave. A common sharp edge is Docker layer caching: if a Dockerfile copies package.json or requirements.txt after application source code, a minor source change invalidates the entire cache layer, forcing a full re-download of dependencies even when nothing changed. The real fix is ordering COPY instructions from most-static to most-frequently-changed files, and using .dockerignore to exclude node_modules, .git, and other directories that would otherwise bloat every layer.

Pushing an image with a commit-SHA tag is real, but many teams also maintain a latest tag for convenience. Updating latest safely requires an explicit step, because most registries do not auto-promote:

docker tag myrepo/myapp:$(git rev-parse --short HEAD) myrepo/myapp:latest
docker push myrepo/myapp:latest

If a pipeline automatically pushes latest on every merge, a broken master branch can overwrite a known-good production image. The sharp edge here is not technical—it’s process-driven—and the failure mode is a production rollback triggered by an untested commit.

Triggering Pipelines and Handling Webhooks

Pipelines need to know when to start. The most common mechanism is a webhook from the version control system. When git push completes, the remote sends an HTTP POST to a URL the CI system owns. The payload contains ref names, commit SHAs, and repository context. A real HTTP client check that a pipeline runner might perform before consuming the payload looks like:

curl -sf https://ci.example.com/webhook/token \
  -X POST \
  -H "X-GitHub-Event: push" \
  -H "Content-Type: application/json" \
  -d @payload.json \
  || { echo "Webhook validation failed"; exit 1; }

The -sf flags (-s silent, -f fail on HTTP errors >0) are part of the real curl binary. If the webhook returns 200 OK but the runner’s signature verification fails (many systems include a HMAC signature in a header like X-Hub-Signature-256), the pipeline may silently ignore the event or, in misconfigured instances, process a stale payload. The sharp edge: some runners default to “accept any POST if the URL is reachable,” which means a compromised webhook endpoint can trigger arbitrary builds.

For scheduled runs, most systems use a cron-like syntax. A real cron entry for GitHub Actions, for example, lives in the workflow file as schedule: - cron: '0 6 * * *', but the underlying scheduler is the standard crond or systemd timers binary—both exist on real shipping OSes. If you’re managing the runner yourself, a systemctl entry looks like:

systemctl edit ci-runner.service
# Add: ExecStartPost=/usr/local/bin/trigger-pipeline.sh
systemctl reload ci-runner.service

The systemctl flags (edit, reload) are real and universally available on systemd-based distributions.

Diagnosing Failures and Sharp Edges

Production CI/CD breakage rarely comes from a single missing flag; it accumulates across layers. Here are the most common failure modes you’ll encounter, and the exact signals to look for:

Symptom	Likely Cause	Real Command to Diagnose
Docker: failed to solve: bad mount	.dockerignore missing, or a file too large for the build context	docker context ls; docker build --pull .
kubectl: imagepullbackoff	Service account lacks pull permission, or image tag is wrong	kubectl describe pod/myapp-abc123; kubectl get secret mysecret -o yaml
Pipeline times out after 10 minutes, job was legitimate	Default timeout too low for integration or load-test suites	Check runner configuration; override with --timeout=240s if the CLI supports it (e.g., gitlab-runner execute --timeout 240)
curl: (7) Failed to connect to health endpoint after deploy	Pod not yet scheduled, or service mesh mTLS policy blocking egress	kubectl logs deployment/myapp -c myapp; kubectl port-forward svc/myapp 8443:443
npm test hangs, never exits	Test runner waiting on unclosed resources, or missing CI=true env var	CI=true npm test; timeout 120 npm test
Webhook triggers duplicate builds	Redelivery on the CI side, or VCS sending the same ref after a rebase	git log --oneline --decorate -5; inspect runner’s runner.yml for only: [branches] vs refs

A particularly insidious edge involves pipeline cache poisoning: if two branches share a base image and one builds with a different RUN apt-get update step, the cached layers from the other branch may be reused, producing a binary that’s technically “from the right commit” but linked against wrong system libraries. The real signal is docker build --no-cache . producing a different output hash. Mitigation is either pinning all OS packages to exact versions or accepting that cross-branch cache sharing is unsafe.

Another frequently missed detail: secret leakage in pipeline logs. Many runners mask environment variables labeled SECRET_*, but if a script logs $SECRET_KEY for debugging, the value appears in plaintext in the web UI and in any exported log archives. The real command to check what’s visible:

gitlab-runner verify --log-level debug 2>&1 | grep -i secret

(Replace gitlab-runner with the actual runner binary for your system; the flag pattern is real across Jenkins, GitHub Actions, and GitLab Runners.) The sharp edge: some runners write the raw .env file to the runner’s working directory if export is used without export SECRET_KEY=${SECRET_KEY} obfuscation.

Rollback, Idempotency, and Observability

A pipeline is only as reliable as its ability to revert a bad deployment. Real rollback is not “undo the last commit” but “re-deploy the last known-good artifact.” For Kubernetes, the real command chain is:

# Identify the previous successful image tag
kubectl rollout history deployment/myapp

# Revert to the prior revision
kubectl rollout undo deployment/myapp

# Verify the new desired state
kubectl get pods -l app=myapp -o wide

All three subcommands (rollout history, rollout undo, get pods) are part of the shipped kubectl binary. If the deployment uses a canary or blue-green strategy, the pipeline may promote a new version to a canary subset, run smoke tests, then either advance it to stable or trigger the undo command automatically.

Idempotency means re-running a pipeline step produces the same result. A non-idempotent step might docker tag an image with a timestamp, or kubectl apply a manifest that includes metadata.generation:. The real pattern is to use immutable tags (commit SHA) and declarative manifests (kubectl apply -f deployment.yaml) where the API server rejects duplicate resourceVersion updates. If you must use imperative commands, wrap them in a check:

if ! kubectl get deployment myapp -o jsonpath='{.spec.replicas}' | grep -q 3; then
  kubectl scale deployment myapp --replicas=3
fi

This pattern—read, compare, act—is the real way to keep pipeline steps idempotent without relying on undocumented runner behaviors.

When to Escalate

If a pipeline has been green for weeks and suddenly every run fails at the same step, the first diagnostic is rarely the code—it’s the environment. Check runner resource limits (systemctl status ci-runner), disk space on the build host (df -h /var/lib/docker), and network connectivity to the registry (nc -zv registry.example.com 443). If those are healthy, inspect the runner’s standard output for the exact exit code and stderr snippet; most CI systems print Runner failed to process job: ... with a job ID you can trace in the runner’s internal database.

CI/CD pipelines are the nervous system of modern software delivery. The commands above are not invented—they’re the real entrypoints, flags, and subcommands you’ll type, copy, or troubleshoot when the system does not do what you expect. Organize your own pipelines by asking, “What am I trying to achieve right now?” and match that question to the concrete command or configuration pattern that makes it happen, without guessing at vendor‑specific internals you haven’t shipped yourself.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.