Vendor Management | The Garnet Wiki

Vendor Management is the disciplined process of selecting, contracting, integrating, monitoring, and evolving external suppliers of technology, services, or products to support engineering and business objectives. It matters when engineering teams rely on third-party systems—APIs, managed services, SaaS platforms, or custom-built integrations—where failure to align on expectations, timelines, and quality leads to delivery delays, integration debt, and operational instability. This page details the core actions, patterns, and pitfalls in managing vendor relationships through the lens of engineering delivery.

Onboarding a New Vendor

Onboarding begins when a team agrees to adopt a new third-party service or product. The goal is to move from a vendor contract to a fully integrated, monitored, and documented dependency in the engineering ecosystem.

Define the Integration Scope

Start with a vendor.yml configuration file in the project root to define the contract:

name: "Payment Gateway"
version: "v1"
api:
  base_url: "https://api.payment-gateway.com/v1"
  auth:
    type: "oauth2"
    client_id: "abc123"
    client_secret: "secure-client-secret"
    token_url: "https://auth.payment-gateway.com/oauth/token"
  endpoints:
    - path: "/charges"
      method: "POST"
      timeout: 2000
    - path: "/refunds"
      method: "POST"
      timeout: 3000
    - path: "/webhooks"
      method: "POST"
      auth: "bearer"

This file is consumed by the CI/CD pipeline and used to generate documentation, test suites, and contract tests. It is the source of truth for the vendor interface.

Set Up Integration Testing

Use curl and jq to verify the vendor’s API contract in a CI job:


curl -s \
  -X POST \
  -H "Authorization: Bearer ${VENDOR_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "amount": 1999,
    "currency": "usd",
    "description": "Monthly subscription"
  }' \
  https://api.payment-gateway.com/v1/charges \
  | jq -e '
    .id | type == "string"
    and .status == "succeeded"
    and .amount == 1999
    and .currency == "usd"
  '

If jq exits non-zero, the test fails. The output includes the full response body and the error message from jq.

Establish Contract Testing

Use Pact or a custom pact verify script to run consumer-driven contract tests. The contract is a JSON file generated by the service consuming the vendor:

{
  "consumer": {
    "name": "Order Service"
  },
  "provider": {
    "name": "Payment Gateway"
  },
  "interactions": [
    {
      "description": "Create charge with valid data",
      "providerState": "a customer exists with id 123",
      "request": {
        "method": "POST",
        "path": "/charges",
        "headers": {
          "Authorization": "Bearer abc123"
        },
        "body": {
          "amount": 1999,
          "currency": "usd",
          "description": "Monthly subscription"
        }
      },
      "response": {
        "status": 201,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "id": "ch_abc123",
          "status": "succeeded",
          "amount": 1999,
          "currency": "usd"
        }
      }
    }
  ]
}

The vendor must implement this contract. The contract is versioned and stored in a shared repository. Each CI run uploads the latest contract to a contracts/ directory in the vendor’s repository.

Monitoring and Observability

Once integrated, the vendor becomes a first-class service. Monitoring ensures that the team can detect and diagnose issues quickly.

Define Key Metrics and Alerts

Use a monitoring.yaml file to specify the metrics to collect and alert on:

services:
  - name: "Payment Gateway"
    endpoints:
      - path: "/charges"
        interval: "5m"
        thresholds:
          - metric: "response_time_p95"
            target: "1.5s"
            alert: "high-latency-charges"
          - metric: "error_rate"
            target: "0.02"
            alert: "high-error-rate"
    health:
      - path: "/health"
        interval: "1m"
        expected_status: 200
        expected_body: { status: "ok" }

This config is consumed by a monitoring tool—such as Prometheus or Grafana—and generates a dashboard.

Build a Health Check Endpoint

The vendor must expose a /health endpoint that returns a JSON object:

{
  "status": "ok",
  "checks": {
    "database": {
      "status": "healthy",
      "details": {
        "latency_ms": 12,
        "connection_count": 15
      }
    },
    "cache": {
      "status": "healthy",
      "details": {
        "hit_rate_percent": 92.4,
        "memory_usage_mb": 450
      }
    },
    "api": {
      "status": "healthy",
      "details": {
        "uptime_hours": 142,
        "request_count_last_hour": 8432
      }
    }
  },
  "timestamp": "2025-04-05T10:30:00Z"
}

The health check is polled every minute by the internal monitoring system. If any check returns status: "unhealthy", an alert is triggered.

Configure Alerting with Thresholds

Alerts use a simple thresholding rule:

High latency: if response_time_p95 exceeds 1.5s for 3 consecutive checks.
High error rate: if error_rate is above 0.02 for 5 minutes.

Alerts are sent via email, Slack, and PagerDuty. Each alert includes the raw response from the health check and the metric time series.

Handling Vendor Outages and Incidents

When a vendor fails, the engineering team must respond quickly and communicate clearly.

Incident Response Workflow

Detect: Monitoring system raises an alert via curl to a webhook endpoint:

{
  "title": "Payment Gateway: High Error Rate",
  "message": "Error rate > 2% over the last 5 minutes.",
  "severity": "warning",
  "timestamp": "2025-04-05T10:35:00Z",
  "details": {
    "error_rate": 0.028,
    "last_5m_requests": 12400,
    "failed_requests": 347
  }
}

Acknowledge: A team member runs:

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"status": "acknowledged", "assignee": "alice"}' \
  https://incidents.company.com/api/v1/incidents/12345/acknowledge

Update: As the incident unfolds, updates are sent:

{
  "incident_id": 12345,
  "status": "in-progress",
  "message": "Database replication lag detected.",
  "timestamp": "2025-04-05T10:40:00Z"
}

Resolve: Once the vendor is stable, the team closes the incident:

curl -X PUT \
  -H "Content-Type: application/json" \
  -d '{"status": "resolved", "resolution": "Database recovery completed."}' \
  https://incidents.company.com/api/v1/incidents/12345/close

Communicate with the Vendor

Use a shared incident-log.txt file in the vendor’s project repository:

2025-04-05T10:30:00Z: [alert] Payment Gateway: Error rate > 2%
2025-04-05T10:35:00Z: [acknowledge] Alice acknowledged incident #12345
2025-04-05T10:40:00Z: [update] Database replication lag detected
2025-04-05T10:55:00Z: [update] Replication catch-up underway
2025-04-05T11:00:00Z: [resolve] Database recovery completed; error rate dropped to 0.9%

The vendor reviews this file daily and uses it to report status updates to their internal teams.

Managing Vendor Evolution and Change

Vendors evolve. Your service must adapt to changes in their APIs, configurations, and behaviors.

Track Vendor Releases

Maintain a releases.json file in the project root:

[
  {
    "version": "1.2.3",
    "released_at": "2025-03-20T08:15:00Z",
    "changes": [
      "Added support for customer metadata in charge creation",
      "Changed error response format for invalid card numbers",
      "Improved rate limiting with burst tokens"
    ],
    "breaks": [
      {
        "field": "metadata",
        "from": "string",
        "to": "object",
        "description": "Previously, metadata was a string; now it’s a JSON object."
      }
    ],
    "deprecations": [
      {
        "field": "customer_id",
        "since": "1.1.0",
        "until": "1.4.0",
        "message": "Use customer_id_v2 instead."
      }
    ]
  }
]

This file is updated after each vendor release and used to generate release notes, update integration tests, and communicate changes to the team.

Apply Changes with Confidence

When upgrading a service to use a new vendor version, follow this workflow:

Verify backward compatibility: run the service against the vendor’s v1.2.3 API with old and new data.
Update configuration in vendor.yml to point to the new version:

api:
  base_url: "https://api.payment-gateway.com/v1"
  version: "1.2.3"
  features:
    - metadata_object: true
    - burst_rate_limiting: true

Run contract tests against the new version.
Deploy with feature flag to gradually roll out changes.

Use a feature-flag.yaml file to control which features are active:

feature_flags:
  metadata_object:
    enabled: true
    rollout: "canary"
    rollout_percentage: 25
  burst_rate_limiting:
    enabled: false
    rollout: "all"

The service reads this file at startup and applies flags at runtime.

Common Pitfalls and Silent Failures

Misconfigured auth: a Bearer token is sent, but the vendor expects API-Key in the X-API-Key header. The response is 401 Unauthorized with no error body. The team assumes the token is valid, but the issue is in the header name.
Unversioned APIs: the vendor introduces a new field, but the team assumes it’s optional. The service crashes when the field is missing, but the error is logged in a logs/ directory with no context.
Outdated contract tests: a new endpoint is added, but the contract test is not updated. The service passes the test, but the endpoint returns a different format than expected. The test passes but the integration fails.
Thresholds too strict: the alert for high latency is set at 1.5s, but the vendor’s P95 is 1.4s. The alert fires, but the issue is not noticed until the next day.
Health check not idempotent: the /health endpoint updates internal state (e.g., marks a check as “recently checked”), so repeated calls produce different results. The monitoring system sees “healthy” when it’s actually “stale”.

Summary

Vendor Management is not a one-time setup. It is an ongoing process of defining, testing, monitoring, and evolving the relationship between engineering teams and external suppliers. It relies on clear contracts, standardized configuration, observable health checks, and disciplined incident workflows. The real success is not just that a vendor works, but that the team knows *how* it works, *why* it fails, and *how to fix it*.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.