Secrets Rotation & Credential Lifecycle

Automate credential rotation. Covers secret lifecycle management, automated rotation patterns, vault integration, zero-downtime rotation, and detecting leaked credentials.

Secrets rotation and credential lifecycle is the practice of managing credentials—API keys, passwords, certificates, tokens, and private keys—throughout their existence: from creation to deployment, usage, rotation, revocation, and eventual retirement. It matters when a system’s security depends on secrets that are not static, especially in distributed, cloud-native environments where services communicate across networks, systems, and teams. Rotation prevents credential compromise, reduces blast radius of breaches, and ensures that access can be quickly revoked or upgraded without downtime.

Planning and Setup

The foundation of effective secrets management is a clear strategy for how credentials are created, stored, and distributed.

Credentials must be versioned, meaning each secret has a unique identifier and timestamp. A new secret is not simply a replacement but a distinct entity. The first step is defining a secret schema: a consistent structure for all secrets, including metadata such as name, type, description, rotation_interval, created_at, expires_at, and status.

Secrets are stored in a centralized secrets store, such as HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or a simple PostgreSQL table. The store must support versioning, listings, and lookup by name. A typical secret entry might look like this in JSON:

{
  "name": "database:prod:read-only",
  "type": "database-connection",
  "value": {
    "host": "db.prod.example.com",
    "port": 5432,
    "database": "analytics",
    "username": "readonly",
    "password": "s3cr3t-4n4lyt1cs"
  },
  "rotation_interval": "7d",
  "created_at": "2024-08-10T12:00:00Z",
  "expires_at": "2025-08-10T12:00:00Z",
  "status": "active"
}

To retrieve a secret, a service calls the secrets store using a standard protocol—typically HTTP with JSON over REST or gRPC. A service might use curl to fetch a secret:

curl -H "Authorization: Bearer ${VAULT_TOKEN}" \
     https://vault.example.com/v1/secrets/database/prod/read-only

The response contains the secret and metadata. The client must cache secrets locally and refresh them on expiry, typically using a Cache-Control header or a TTL-based fetch.

Creating Secrets

Creating a secret involves generating a new credential pair (e.g., a username and password, or a client ID and secret), storing it in the secrets store, and marking it as active.

A new secret is created with a rotation interval—the duration after which it should be replaced. The default is 7 days, but some systems rotate weekly, others daily or quarterly.

The process begins with a generation script or a CLI command. For instance:

vault write secrets/database/prod/read-only \
  host=db.prod.example.com \
  port=5432 \
  database=analytics \
  username=readonly \
  password=super-secret-pw \
  rotation_interval=7d \
  description="Read-only access for analytics service"

The store returns the secret with a unique version identifier, such as v1, v2, etc. The secret is now active, and services can start using it.

Key pitfalls:

Deploying Secrets

Once created, secrets must be deployed to services. Deployment means writing the secret to the service’s configuration, restarting or reloading, and verifying availability.

Services use a configuration file, environment variables, or a runtime API to access secrets.

For a service using environment variables, a deployment script might do:

export DATABASE_URL="postgresql://readonly:[email protected]:5432/analytics"
export AWS_ACCESS_KEY_ID="AKIA1234567890"
export AWS_SECRET_ACCESS_KEY="s3cr3t-k3y-1234567890"

Alternatively, services use a configuration file such as config.yaml:

database:
  url: "postgresql://readonly:[email protected]:5432/analytics"
  pool_size: 10
  connection_timeout: 30

aws:
  access_key_id: "AKIA1234567890"
  secret_access_key: "s3cr3t-k3y-1234567890"
  region: "us-west-2"

Services can also poll for secrets using a /secrets endpoint:

GET /secrets HTTP/1.1
Host: service.example.com
Accept: application/json
Authorization: Bearer <token>

Response:

{
  "secrets": {
    "database": {
      "url": "postgresql://readonly:[email protected]:5432/analytics"
    },
    "aws": {
      "access_key_id": "AKIA1234567890",
      "secret_access_key": "s3cr3t-k3y-1234567890"
    }
  },
  "version": "v2",
  "last_updated": "2024-08-10T15:30:00Z"
}

Critical failure mode:

Rotating Secrets

Rotation is the periodic replacement of a secret with a new version. It is the core of the credential lifecycle.

Rotation occurs when:

The process is atomic and idempotent:

  1. Generate a new secret version (v2).
  2. Activate v2 as active.
  3. Wait for services to pick up v2.
  4. Deactivate v1 as inactive.
  5. Optionally, revoke v1 (e.g., by invalidating its password or token).

Rotation is often asynchronous. A service might fetch the latest secret every 30 seconds, but it takes 15 minutes for all services to adopt v2.

A rotation script might look like:


vault write secrets/database/prod/read-only \
  host=db.prod.example.com \
  port=5432 \
  database=analytics \
  username=readonly-v2 \
  password=very-secure-pw-v2 \
  rotation_interval=7d \
  version="v2"

# Step 2: Mark v2 as active
vault write secrets/database/prod/read-only/active \
  version="v2"

# Step 3: Schedule deactivation of v1
vault write secrets/database/prod/read-only/deactivate \
  version="v1" \
  reason="new rotation"

Common failure modes:

Revoking and Retiring Secrets

Revocation removes access for a secret, even when it is still active. It is used during breaches, migrations, or decommissioning.

To revoke a secret, the secrets store marks it as revoked and invalidates its credentials.

For example, to revoke a database secret:

POST /secrets/database/prod/read-only/revoke
Content-Type: application/json

{
  "version": "v1",
  "reason": "Security incident: leaked credentials",
  "effective_at": "2024-08-11T10:00:00Z"
}

The store updates the secret’s status to revoked and invalidates the password or token.

Services must check revocation status before using a secret. A common pattern is to include a revoked_at timestamp and a revoked_by field.

When a service connects to a database, it checks the credentials against a revocation list:

SELECT * FROM secrets_revoked 
WHERE secret_name = 'database:prod:read-only' 
  AND version = 'v1'
  AND effective_at <= NOW();

If a secret is revoked, the service rejects the connection and logs:

ERROR: Secret database:prod:read-only v1 is revoked (reason: Security incident: leaked credentials)

Key insights:

Monitoring and Troubleshooting

A secrets lifecycle is only as strong as its monitoring.

Critical metrics include:

A monitoring script checks for expired secrets:

for secret in $(curl -s http://vault.example.com/v1/secrets | jq -r '.data.keys[]'); do
  version=$(curl -s http://vault.example.com/v1/secrets/$secret | jq -r '.data.version')
  rotation_interval=$(curl -s http://vault.example.com/v1/secrets/$secret | jq -r '.data.rotation_interval')
  created_at=$(curl -s http://vault.example.com/v1/secrets/$secret | jq -r '.data.created_at')

  # Parse duration: e.g. "7d" → 7 days
  interval_days=$(echo $rotation_interval | sed 's/d$//')
  created_time=$(date -d "$created_at" +%s)
  now=$(date +%s)
  elapsed=$(( (now - created_time) / 86400 ))  # seconds to days

  if [ $elapsed -gt $interval_days ]; then
    echo "WARNING: $secret (v$version) is due for rotation"
  fi
done

Failure modes:

Summary

Secrets rotation and credential lifecycle is a continuous, automated process: create, deploy, rotate, revoke, retire. It ensures that credentials are not static, but dynamic, trusted, and auditable. The success of a system depends not just on having secrets, but on knowing when and how they change—and how services adapt to those changes.

This page was rewritten on 16 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.