GitOps Automation: Git as the Single Source of Truth for Operations

Implement GitOps workflows where every operational change is a pull request. Covers ArgoCD, Flux, repository structure, drift detection, secret management, and progressive delivery patterns for Kubernetes-native automation.

GitOps Automation: Git as the Single Source of Truth for Operations

Git is the central system of record for infrastructure, application configuration, and operational policies. When every change to the running environment is expressed as a commit in a Git repository—whether it's a Kubernetes manifest, a CI/CD pipeline definition, or a compliance policy—then the repository becomes the single source of truth for operations. This approach matters when teams need to audit, roll back, scale, or replicate environments with confidence, and when multiple teams are deploying services independently but must coordinate around shared infrastructure.

Defining the Desired State

The foundation of GitOps is the desired state of the system, encoded as declarative configuration files stored in Git. These files describe how the system should look at any point in time, and every change to this state is versioned, reviewed, and tested.

A typical desired state consists of:

For example, a kubernetes/web-app.yaml file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  namespace: production
spec:
  replicas: 4
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: app
        image: myregistry/web-app:v1.4.2
        ports:
        - containerPort: 8080
        envFrom:
        - configMapRef:
            name: app-config
        - secretRef:
            name: app-secrets

This manifest is committed, pushed, and merged into main branch. The GitOps system then detects the change and applies it to the cluster.

Synchronizing the System with Git

When a Git repository is the source of truth, the system must continuously reconcile the actual state of the environment with the desired state in Git. This reconciliation is driven by a sync loop that polls the Git repository and applies changes.

The sync loop is typically implemented as a periodic job or long-running process. It:

  1. Fetches the latest state of the Git repository.
  2. Evaluates the desired state from the current commit.
  3. Applies the desired state to the target environment using standard tools.
  4. Reports the result back to Git as a status.

A typical sync loop using kubectl and git:

#!/bin/bash
set -euo pipefail

REPO_DIR="/var/gitops/repo"
GIT_REMOTE="https://github.com/example/infra.git"
BRANCH="main"


cd $REPO_DIR
git fetch origin $BRANCH
git checkout $BRANCH

# Apply all manifests in kubernetes/
kubectl apply -f kubernetes/ --dry-run=client

# Apply in production
kubectl apply -f kubernetes/ -n production

# Update Git with sync status
git add .
git commit -m "Sync: applied changes from $(git rev-parse HEAD)"
git push origin $BRANCH

# Write sync status to a file in the repo
cat > sync-status.json <<EOF
{
  "commit": "$(git rev-parse HEAD)",
  "timestamp": "$(date -Iseconds)",
  "status": "success",
  "applied": [
    "kubernetes/web-app.yaml",
    "kubernetes/db.yaml"
  ]
}
EOF

git add sync-status.json
git commit -m "Update sync status"
git push origin $BRANCH

The sync loop runs every 5 minutes via cron or as a container in Kubernetes.

Managing Changes: Pull Requests and Review

Every change to the operational state is a Git pull request (PR). The PR includes:

A PR to update the web app’s image version:

diff --git a/kubernetes/web-app.yaml b/kubernetes/web-app.yaml
index 1a2b3c4..5d6e7f8 100644
--- a/kubernetes/web-app.yaml
+++ b/kubernetes/web-app.yaml
@@ -10,7 +10,7 @@ spec:
   replicas: 4
   selector:
     matchLabels:
-      app: web-app
+      app: web-app-v2
   template:
     metadata:
       labels:
@@ -22,7 +22,7 @@ spec:
         image: myregistry/web-app:v1.4.3
         ports:
         - containerPort: 8080
-        envFrom:
+        envFrom:
         - configMapRef:
             name: app-config
         - secretRef:

The CI system runs kubectl apply --dry-run and validates the manifest:

kubectl apply -f kubernetes/ --dry-run=client -o yaml | tee /tmp/dry-run.yaml

If the dry run fails, the PR status is marked as failed, and the reviewer sees the error in the PR UI.

Handling Rollbacks and Drift

When a production issue arises, the team can roll back to a previous state by simply merging a PR that reverts a previous commit.

For example, to revert a deployment:

git revert --no-commit HEAD~2
git commit -m "Rollback web-app to v1.4.1"
git push origin main

The sync loop detects the new commit and applies the reverted configuration.

Drift—deviation between desired state in Git and actual state in the cluster—is detected by comparing the current state against the desired state.

A drift detector script:

#!/bin/bash
set -euo pipefail

TEMP_DIR=$(mktemp -d)
cd $TEMP_DIR

# Get current state of production
kubectl get all -n production -o yaml > current.yaml

# Get desired state from Git
git clone https://github.com/example/infra.git desired
cp desired/kubernetes/* current.yaml

# Compare using kubectl diff
kubectl diff -f current.yaml -f desired/kubernetes/ --context=production

# Report drift
diff -u desired/kubernetes/ current.yaml | tee drift.diff

The output drift.diff shows missing, updated, or missing resources. This report is committed back to Git as part of the sync loop.

Automating Policy Enforcement

GitOps extends beyond configuration: it enforces policies through automated checks.

A policy is a declarative rule that must be satisfied for a change to be accepted. Policies are stored as files in policies/ and evaluated during CI and sync.

Example policy: policies/require-annotations.yaml

package policy

default require_annotations = false

require_annotations {
  input.resources[_].metadata.annotations["team"]
  input.resources[_].metadata.annotations["owner"]
}

This policy is evaluated using opa (Open Policy Agent):

opa build -o policy.tar.gz policies/
opa run -w 5s --set input.resources=$(kubectl get all -n production -o json) \
  --set input.context=production \
  --server.policy policy.tar.gz \
  --server.port 8181

The result is a JSON file with policy violations:

{
  "result": {
    "require_annotations": {
      "result": false,
      "explanation": [
        "Resource: Deployment/web-app in namespace production is missing 'owner' annotation"
      ]
    }
  }
}

This output is uploaded as a status check on the PR. If any policy fails, the PR is blocked.

Handling Failures and Feedback

Common failure modes in GitOps automation:

A common misstep is confusing --dry-run=client with --dry-run=server. The former validates the manifest but does not apply it; the latter applies it to the cluster and returns the applied state. Using --dry-run=client in a sync loop leads to unnoticed application failures.

Another edge: when a new cluster is bootstrapped, the sync loop must initialize the cluster from the Git repository. This requires:

Example bootstrap:

git clone https://github.com/example/infra.git /tmp/bootstrap
cd /tmp/bootstrap
kubectl apply -f bootstrap.yaml
kubectl apply -f kubernetes/

The sync loop runs from the first commit, ensuring no configuration is missed.

Tracking the State of Automation

GitOps automation is not only about applying changes—it is about visibility into the entire operational lifecycle.

The following files are used to track the automation state:

These files are committed and pushed to Git, creating a full audit trail.

A status command for the automation system:

#!/bin/bash
set -euo pipefail

echo "=== GitOps Sync Status ==="
cat sync-status.json

echo -e "\n=== Drift Report ==="
cat drift.diff

echo -e "\n=== Policy Evaluation ==="
jq -C . policy-results.json

This output is logged during the sync loop and can be viewed in the CI system.

The Role of Git in Operational Decision-Making

When a service breaks, operators can answer:

The answer is always in Git.

To investigate a failure:

git log --oneline --graph --decorate --all --since="2 hours ago"
git show 1a2b3c4
git diff HEAD~1 HEAD

The change history is the operational history.

Each merge commit is a change event. The PR title, description, and review comments become the operational documentation.

When an incident occurs, the team creates a notes/2024-05-13-incident.md file in the repo:

# Incident: Web App Slowness

- **Date**: 2024-05-13
- **Root Cause**: High memory usage in web-app pods
- **Fix**: Increased replica count from 4 to 6
- **Change**: Merged PR #342: "Increase web-app replicas"

## Affected Resources
- Deployment/web-app
- Service/web-app
- ConfigMap/app-config

## Associated Git Commits
- 1a2b3c4: Increase replicas
- 5d6e7f8: Update app image to v1.4.3

This file is committed and merged into main, becoming part of the system's history.

GitOps turns operational work into a continuous, versioned, and auditable process. Every change, every policy, every failure is a commit. The repository is not just a codebase—it is the living record of the entire system’s operations.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.