Developer Experience Metrics

Production engineering guide for developer experience metrics covering patterns, implementation strategies, and operational best practices.

Developer experience metrics track the real-time health, efficiency, and satisfaction of software developers using a platform. They are essential when engineering teams grow beyond a few teams and need to measure how well the platform enables feature delivery, debugging, and onboarding. These metrics become critical when developers report friction, onboarding takes weeks, and release cycles are inconsistent.

Measuring Developer Onboarding Time

Onboarding time begins when a new engineer receives their first task and ends when they successfully deploy their first commit to production. The key metric is *time to first deploy*, measured from the moment a developer clones the repository to the moment their code is live in production.

To capture this, developers run a ./scripts/onboarding-checklist.sh script on their machine:

#!/bin/bash

set -euo pipefail

echo "Starting onboarding checklist..."

# Validate environment setup
if ! command -v node &>/dev/null; then
  echo "❌ Node.js not installed"
fi

if ! command -v docker &>/dev/null; then
  echo "❌ Docker not installed"
fi

# Validate CI/CD pipeline
if ! git rev-parse --short HEAD | grep -q "$(curl -s http://ci.example.com/latest-build | jq -r '.commit_sha')"; then
  echo "❌ Local commit not in CI pipeline"
fi

# Validate deployment
if ! curl -s http://app.example.com/health | jq -e '.status == "UP"'; then
  echo "❌ App not running in production"
fi

echo "✅ Onboarding checklist complete"

This script is run as part of a make onboarding command. The build system logs the start and end timestamps of each make onboarding run in a database table:

CREATE TABLE onboarding_metrics (
  id BIGSERIAL PRIMARY KEY,
  developer_id UUID NOT NULL,
  start_time TIMESTAMPTZ NOT NULL,
  end_time TIMESTAMPTZ,
  success BOOLEAN NOT NULL,
  error_messages TEXT[],
  git_commit_sha TEXT NOT NULL,
  duration_seconds INTERVAL
);

A duration_seconds value under 30 minutes is considered ideal. A failure occurs when error_messages contains "Node.js not installed" and duration_seconds > 60, indicating a bottleneck in tooling setup.

Tracking Continuous Integration Effectiveness

The primary metric for CI effectiveness is *mean time to merge (MTTM)* — the average time from when a pull request is opened to when it is merged. This includes time spent in review, test failures, and manual approval steps.

CI pipelines are triggered by git push and report status via HTTP endpoints. Each job emits a status.json file in the artifacts/ directory:

{
  "job_id": "ci-2023-09-14-1234",
  "status": "success",
  "started_at": "2023-09-14T12:00:00Z",
  "completed_at": "2023-09-14T12:18:30Z",
  "duration_seconds": 1050,
  "steps": [
    {
      "name": "test",
      "status": "success",
      "duration_seconds": 600,
      "output": [
        "Running 23 tests...",
        "✓ All tests passed"
      ]
    },
    {
      "name": "build",
      "status": "failed",
      "duration_seconds": 250,
      "output": [
        "Building with webpack...",
        "ERROR: Cannot find module 'lodash'"
      ]
    },
    {
      "name": "lint",
      "status": "success",
      "duration_seconds": 200,
      "output": [
        "Linting 14 files...",
        "No issues found"
      ]
    }
  ]
}

A PR is considered to have failed if any step has status: "failed" and output contains "ERROR:". The CI system logs each job to a ci_jobs table:

CREATE TABLE ci_jobs (
  id SERIAL PRIMARY KEY,
  pr_id INT NOT NULL,
  job_id TEXT NOT NULL,
  status TEXT NOT NULL,
  started_at TIMESTAMPTZ NOT NULL,
  completed_at TIMESTAMPTZ,
  duration_seconds INTERVAL,
  metadata JSONB
);

The metadata field contains the status.json output. The MTTM is computed as the average completed_at - started_at across all ci_jobs where pr_id matches the PR number.

A silent failure occurs when a test step passes locally but fails in CI due to missing environment variables. The CI system does not log the environment variables in metadata, so a missing NODE_ENV=production in a docker-compose.yml file causes test flakiness that goes unnoticed.

Monitoring Developer Workflow Through the Platform

When developers create, run, and manage environments, the key metric is *environment creation latency* — the time from when a developer requests a new environment to when it becomes available.

A developer runs:

kubectl create -f - <<EOF
apiVersion: platform.example.com/v1
kind: EnvironmentRequest
metadata:
  name: feature-user-profiles
spec:
  base: "dev"
  size: "medium"
  services:
    - name: auth
      image: registry.example.com/auth:latest
    - name: profile
      image: registry.example.com/profile:latest
EOF

The platform responds with a 201 Created and a location header pointing to the environment’s status endpoint:

HTTP/1.1 201 Created
Location: /api/environments/12345
Content-Type: application/json

{
  "id": "12345",
  "status": "pending",
  "requested_at": "2023-09-14T13:00:00Z",
  "requested_by": "[email protected]"
}

The environment is considered ready when status is ready and last_updated_at is within 5 minutes of requested_at. A status of failed includes a failure_reason string and error_log array.

The platform maintains a environment_lifecycle table:

CREATE TABLE environment_lifecycle (
  id SERIAL PRIMARY KEY,
  environment_id TEXT NOT NULL,
  state TEXT NOT NULL,
  timestamp TIMESTAMPTZ NOT NULL,
  details JSONB
);

When a developer runs kubectl get environments feature-user-profiles, the command waits until state = 'ready' and logs the duration_seconds between requested_at and timestamp.

A common failure mode occurs when the services array is misformatted in the EnvironmentRequest. The API expects a services: [ { name, image } ] but accepts services: { name: "auth", image: "..." }. This leads to inconsistent behavior: some environments have only one service, others have many, but the configuration is not validated during creation.

Assessing Developer Satisfaction and Pain Points

Developer satisfaction is measured through *incident response time* and *error resolution time*.

When a developer encounters a problem, they report it via a report-error script:

./scripts/report-error.sh \
  --title "500 error on /api/users" \
  --context "Deployed via CI, environment: feature-user-profiles" \
  --stack-trace "$(cat logs/2023-09-14.log)" \
  --tags "api", "user", "error"

This script sends a POST /api/errors request:

POST /api/errors HTTP/1.1
Host: platform.example.com
Content-Type: application/json

{
  "title": "500 error on /api/users",
  "context": "Deployed via CI, environment: feature-user-profiles",
  "stack_trace": [
    "Error: Internal Server Error",
    "    at UserRouter.getUsers (src/routes/users.js:25:15)",
    "    at Layer.handle [as handle_request] (node_modules/express/lib/router/layer.js:95:5)",
    "    at next (node_modules/express/lib/router/route.js:137:13)"
  ],
  "tags": ["api", "user", "error"],
  "reported_at": "2023-09-14T13:45:30Z",
  "reported_by": "[email protected]"
}

The platform logs each error in a developer_errors table:

CREATE TABLE developer_errors (
  id SERIAL PRIMARY KEY,
  title TEXT NOT NULL,
  context TEXT,
  stack_trace TEXT[],
  tags TEXT[],
  reported_at TIMESTAMPTZ NOT NULL,
  reported_by UUID NOT NULL,
  resolved_at TIMESTAMPTZ,
  resolution_notes TEXT,
  status TEXT NOT NULL -- "new", "in-progress", "resolved"
);

The *error resolution time* is the average resolved_at - reported_at. A status of in-progress includes a assignee field and last_updated timestamp.

A subtle failure occurs when stack_trace is not properly serialized: if stack_trace is an array of strings but the API expects a single string with \n between lines, errors appear with truncated traces. The platform does not validate the structure, so developers often report errors with incomplete context.

Evaluating Platform Self-Service and Autonomy

The metric *percentage of self-service tasks* measures how often developers complete a workflow without help from platform engineers.

A self-service task is one that a developer completes end-to-end using only the platform’s CLI, API, and documentation. The platform tracks this via task_completion events:

{
  "task_id": "deploy-2023-09-14",
  "type": "deployment",
  "status": "completed",
  "started_at": "2023-09-14T14:00:00Z",
  "completed_at": "2023-09-14T14:15:00Z",
  "steps": [
    {
      "name": "build",
      "status": "success",
      "duration_seconds": 600,
      "output": "Built with webpack"
    },
    {
      "name": "test",
      "status": "success",
      "duration_seconds": 300,
      "output": "Ran 23 tests, all passed"
    },
    {
      "name": "deploy",
      "status": "failed",
      "duration_seconds": 150,
      "output": "Deploy failed: service 'auth' not reachable"
    }
  ],
  "user_id": "[email protected]",
  "metadata": {
    "environment": "feature-user-profiles",
    "ci_job": "ci-2023-09-14-1234"
  }
}

The platform computes the self-service rate as the ratio of tasks with status: "completed" and steps where all status: "success".

A common confusion is between *success* and *completion*. A task is marked completed even if some steps fail, but only tasks with all steps success are considered fully self-service.

Another failure mode is the misalignment between metadata and actual workflow. When a developer runs make deploy and the script logs environment: dev but the metadata field contains environment: staging, the task is not counted as self-service, even though the developer intended it.

Tracking Golden Path Adoption

The golden path is the recommended way to do something on the platform. The metric *golden path adoption rate* measures how often developers use it instead of custom or ad-hoc workflows.

The platform defines golden paths via golden-path.yaml files in the repository root:

version: 1
name: deploy-to-production
description: "Deploy a release to production using CI and a feature flag"
steps:
  - name: "build"
    command: "make build"
    expected_exit_code: 0
    timeout_seconds: 300
  - name: "test"
    command: "make test"
    expected_exit_code: 0
    timeout_seconds: 600
  - name: "deploy"
    command: "kubectl apply -f manifests/production.yaml"
    expected_exit_code: 0
    timeout_seconds: 900
  - name: "feature-flag"
    command: "curl -X POST http://flags.example.com/api/flags/enable -d '{\"name\": \"new-ui\", \"environment\": \"prod\"}'"
    expected_exit_code: 0
    timeout_seconds: 300

The platform monitors which golden-path.yaml files are used and logs each execution to golden_path_executions:

CREATE TABLE golden_path_executions (
  id SERIAL PRIMARY KEY,
  path_id TEXT NOT NULL,
  started_at TIMESTAMPTZ NOT NULL,
  completed_at TIMESTAMPTZ,
  status TEXT NOT NULL, -- "success", "failed"
  errors TEXT[],
  metadata JSONB
);

The adoption rate is the percentage of golden_path_executions where path_id matches the canonical name and status = "success".

A silent failure occurs when metadata is not updated after a successful execution. The platform does not record the environment or release_version, so ad-hoc deployments are not tracked, and the adoption rate is inflated by unrecorded runs.

A developer may run make deploy and assume it follows the golden path, but the makefile does not invoke golden-path.yaml — the platform does not enforce it. The metric only captures the intentional use of the golden path, not the implicit one.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.