SOC 2 Type II Compliance Engineering

SOC 2 Type II Compliance Engineering is the disciplined practice of designing, implementing, and maintaining systems that continuously demonstrate adherence to the five trust service criteria—security, availability, processing integrity, confidentiality, and privacy—over a sustained period, typically six to twelve months. It matters when auditors require evidence that controls are not just documented but *operational* and *effective* across time, not just at a point in time.

Designing the Control Framework

Start by defining the control objectives for each trust service criterion. For each, specify the control, its owner, frequency, and evidence format. A control for security might be:

• Control: Network firewalls are configured to enforce minimum access policies.
• Owner: Infrastructure team.
• Frequency: Continuous (daily), with monthly validation.
• Evidence: Firewall rule configuration files, access logs, and audit trails.

Use a central configuration repository—typically a git repository—to model this. Define a controls/ directory with one YAML file per control.

control_id: S-1
title: "Enforce minimum firewall rules"
description: "All production services must have at least the following rules applied: SSH (22), HTTPS (443), and internal database (5432)."
owner: "infrastructure"
frequency: "daily"
evidence:
  - type: "file"
    path: "/etc/firewall/rules.v4"
    format: "iptables"
    expected: ["ssh", "https", "db"]
  - type: "log"
    source: "firewall-audit"
    query: |
      SELECT action, src_ip, dst_port
      FROM audit_logs
      WHERE timestamp > NOW() - INTERVAL '1 day'
      AND event_type = 'firewall_rule_applied'

This structure enables automation and traceability. When a control fails, the system knows not only *what* failed but *why* and *where*.

Automating Evidence Collection

Evidence must be collected automatically and persisted in a structured format. Use systemd services to run daily collection tasks, triggered by cron.

# /etc/cron.d/evidence-collection
0 2 * * * root /opt/compliance/bin/collect-evidence --control S-1 --output /var/lib/compliance/evidence/s-1-$(date +\%Y-\%m-\%d).json

The collect-evidence binary reads the control definition, executes the required checks, and emits a JSON file with metadata and results.

{
  "control_id": "S-1",
  "timestamp": "2024-01-15T02:00:00Z",
  "status": "passed",
  "evidence": [
    {
      "type": "file",
      "path": "/etc/firewall/rules.v4",
      "content": [
        "INPUT -i eth0 -p tcp --dport 22 -j ACCEPT",
        "INPUT -i eth0 -p tcp --dport 443 -j ACCEPT",
        "INPUT -i eth0 -p tcp --dport 5432 -j ACCEPT"
      ],
      "valid": true,
      "missing": [],
      "extra": []
    },
    {
      "type": "log",
      "source": "firewall-audit",
      "query": "SELECT action, src_ip, dst_port FROM audit_logs WHERE timestamp > NOW() - INTERVAL '1 day' AND event_type = 'firewall_rule_applied'",
      "records": [
        {
          "action": "applied",
          "src_ip": "192.168.1.1",
          "dst_port": 22,
          "timestamp": "2024-01-15T01:30:12Z"
        }
      ],
      "duration": "PT1H23M",
      "count": 123
    }
  ],
  "notes": [
    "Rule applied via Ansible playbook on 2024-01-14T23:00:00Z"
  ]
}

The collect-evidence command fails silently if any expected file is missing or if a query returns fewer than expected records. It logs to journalctl and exits with code 1 on failure.

Validating Control Effectiveness

Validation checks verify that controls are not only present but *working*. Use a validate-control binary that runs periodic health checks.

# /etc/systemd/system/[email protected]
[Unit]
Description=Validate SOC 2 control %i
After=network.target

[Service]
Type=oneshot
ExecStart=/opt/compliance/bin/validate-control --control %i --report /var/lib/compliance/reports/%i-%Y-%m-%d.json
StandardOutput=journal
StandardError=journal
User=compliance

[Install]
WantedBy=multi-user.target

This service runs once per day for each control and produces a report with detailed diagnostics.

{
  "control_id": "PI-3",
  "timestamp": "2024-01-15T03:00:00Z",
  "status": "warning",
  "checks": [
    {
      "name": "Data retention policy met",
      "result": "failed",
      "details": {
        "expected_days": 365,
        "actual_days": 342,
        "threshold": 350,
        "missing": [
          "user-23198-2023-12-01.json.gz",
          "user-23198-2023-12-02.json.gz"
        ]
      }
    },
    {
      "name": "Encryption in transit verified",
      "result": "passed",
      "details": {
        "protocol": "TLSv1.3",
        "ciphers": [
          "TLS_AES_256_GCM_SHA512",
          "TLS_AES_128_GCM_SHA256"
        ],
        "verified": true
      }
    }
  ],
  "summary": "3 of 4 checks passed. 342 days of data retained, 2 files missing."
}

A control is marked *passed* only if all checks pass; *warning* if one or more checks fail but not critically; *failed* if any critical check fails.

Managing the Audit Trail

Every control action must be logged to a central audit trail. Use a PostgreSQL database to store audit events with structured metadata.

CREATE TABLE audit_events (
    id BIGSERIAL PRIMARY KEY,
    control_id TEXT NOT NULL,
    event_type TEXT NOT NULL,
    timestamp TIMESTAMPTZ NOT NULL,
    source TEXT,
    payload JSONB,
    status TEXT CHECK (status IN ('success', 'failure', 'warning', 'skipped')),
    duration_ms BIGINT,
    triggered_by TEXT,
    correlation_id TEXT
);

Each collect-evidence and validate-control run inserts an event:

INSERT INTO audit_events (
    control_id,
    event_type,
    timestamp,
    source,
    payload,
    status,
    duration_ms,
    triggered_by,
    correlation_id
) VALUES (
    'S-1',
    'evidence-collected',
    '2024-01-15T02:00:00Z',
    'systemd/cron',
    '{"evidence_files": ["/etc/firewall/rules.v4"], "log_entries": 123}',
    'success',
    1423,
    'cron-daily',
    'c7e1a8b2-3c4d-5e6f-7a8b-9c0d1e2f3a4b'
);

Auditors use this trail to trace how control evidence evolved over time. A failure in a control is not just a single log line but a lineage of events, each with its own metadata and payload.

Synchronizing Evidence Across Systems

Evidence is not static. It must be synchronized across systems—especially when control owners use different tools. Use rsync with --checksum and --delete to ensure consistency.

# /opt/compliance/bin/sync-evidence.sh
#!/bin/bash
set -euo pipefail

SOURCE="/var/lib/compliance/evidence/"
DEST="evidence-storage:/evidence/"
LOG="/var/log/compliance/sync.log"

rsync \
  --archive \
  --checksum \
  --delete \
  --verbose \
  --log-file="$LOG" \
  "$SOURCE" "$DEST"

# Verify that all files in evidence were synced
if ! find "$SOURCE" -type f -name "*.json" | xargs md5sum | sort > /tmp/evidence-md5s.txt; then
  echo "Error: MD5 verification failed" >&2
  exit 1
fi

# Verify against destination
md5sum -c /tmp/evidence-md5s.txt < <(
  ssh evidence-storage "find /evidence -type f -name '*.json' -exec md5sum {} +"
)

This script runs nightly and logs output to journalctl. A missing file or checksum mismatch triggers an alert.

Handling Control Exceptions and Rationale

Controls may be exceptions—e.g., a service is temporarily unavailable, so a control is deferred. Use a exceptions/ directory in git to track these.

# exceptions/2024-01-10-s-1.yml
control_id: S-1
start_date: "2024-01-10T00:00:00Z"
end_date: "2024-01-14T23:59:59Z"
reason: "Network migration in progress. Firewalls being updated across data centers."
approved_by: "[email protected]"
evidence: [
  "https://docs.company.com/network-migration-plan#phase-2",
  "https://logs.company.com/firewall-updates-2024-01-10"
]

When a control is validated during an exception period, the validation script checks exceptions/ and includes the exception metadata in the report.

Ensuring Evidence Integrity and Provenance

Evidence must be immutable and verifiable. Use git to version evidence files and detached signatures with gpg.

# /opt/compliance/bin/commit-evidence.sh
#!/bin/bash
set -euo pipefail

cd /var/lib/compliance/evidence

git add .
git commit -m "Evidence collection for $(date +'%Y-%m-%d')"
git tag -a "evidence-$(date +'%Y-%m-%d')" -m "Daily evidence collection"

# Sign the commit
git gpg --detach-sign HEAD

# Push to remote
git push origin main --tags

Auditors can verify the integrity of evidence by checking the git history and verifying GPG signatures:

git verify-commit HEAD
# Output:
# gpg: Signature made Thu Jan 18 08:12:34 2024 UTC
# gpg:                using RSA key 1234567890ABCDEF
# gpg: Good signature from "Compliance Team <[email protected]>"
# gpg: WARNING: Signature not yet verified

A failure here implies that evidence was tampered with or collected incorrectly.

Detecting and Alerting on Control Failures

Use systemctl to run a check-controls service that runs every 15 minutes.

# /etc/systemd/system/check-controls.service
[Unit]
Description=Check SOC 2 control status
After=network.target

[Service]
Type=oneshot
ExecStart=/opt/compliance/bin/check-controls --report /var/lib/compliance/health.json
StandardOutput=journal
StandardError=journal
User=compliance

[Timer]
OnCalendar=minute=0,15,30,45
Persistent=true

[Install]
WantedBy=timers.target

The check-controls binary queries the audit trail for recent control validations and emits a health report.

{
  "summary": {
    "total_controls": 12,
    "passed": 9,
    "warning": 2,
    "failed": 1
  },
  "details": [
    {
      "control_id": "PI-3",
      "status": "failed",
      "last_run": "2024-01-15T03:00:00Z",
      "failure_reason": "Data retention below threshold: 342 days, expected 365.",
      "events": [
        {
          "type": "evidence-collected",
          "timestamp": "2024-01-15T02:00:00Z",
          "status": "success"
        },
        {
          "type": "validate-control",
          "timestamp": "2024-01-15T03:00:00Z",
          "status": "failed",
          "details": {
            "check": "Data retention policy met",
            "actual": 342,
            "expected": 365
          }
        }
      ]
    }
  ]
}

This report is consumed by a monitoring system—e.g., Prometheus—which alerts on control failures via Alertmanager.

The Failure Modes You Must Know

• Evidence file missing: collect-evidence runs, but the expected file is not present. The process exits 1, but no alert is generated because systemd does not track exit codes.
• Checksum mismatch during sync: rsync logs but does not fail on md5sum check. Solution: use set -e and trap in sync-evidence.sh.
• Validation query returns fewer records than expected: the validate-control binary compares actual vs. expected counts, but the query is not idempotent. Fix: add ORDER BY and LIMIT to query, and log execution time.
• Control exception not applied: check-controls runs, but the control is marked as active even during an exception. The system must check exceptions/ and apply exception logic before determining status.
• Audit trail event missing: collect-evidence logs to journalctl, but the audit_events table is not updated. Cause: psql command failed silently due to missing credentials or network outage.

Final Output: The Compliance Package

At the end of each quarter, the compliance engineer generates a compliance package by gathering all evidence, validation reports, audit trail entries, and exceptions.

# /opt/compliance/bin/generate-compliance-package.sh
#!/bin/bash
set -euo pipefail

OUTPUT_DIR="/var/lib/compliance/packages/q1-2024"
mkdir -p "$OUTPUT_DIR"

# Copy all evidence files
cp -r /var/lib/compliance/evidence "$OUTPUT_DIR/evidence"

# Copy reports
cp /var/lib/compliance/reports/*.json "$OUTPUT_DIR/reports/"

# Export audit trail to CSV
psql -c "
  COPY (
    SELECT control_id, event_type, timestamp, source, status, duration_ms
    FROM audit_events
    WHERE timestamp >= '2024-01-01T00:00:00Z'
    AND timestamp < '2024-04-01T00:00:00Z'
    ORDER BY timestamp
  ) TO STDOUT WITH CSV HEADER
" > "$OUTPUT_DIR/audit-trail.csv"

# Generate a summary JSON
cat > "$OUTPUT_DIR/summary.json" <<EOF
{
  "quarter": "Q1 2024",
  "controls": {
    "total": 12,
    "passed": 9,
    "warning": 2,
    "failed": 1
  },
  "exceptions": 3,
  "evidence_files": 142,
  "last_run": "2024-01-15T03:00:00Z"
}
EOF

# Create a tarball
tar -czf "$OUTPUT_DIR.tar.gz" -C "$OUTPUT_DIR" .

This package is the final deliverable for auditors: a complete, verifiable, and time-anchored record of SOC 2 Type II compliance.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.