Cloud Serverless Patterns

Production engineering guide for cloud serverless patterns covering patterns, implementation strategies, and operational best practices.

Cloud serverless patterns enable developers to build and operate applications without managing servers, focusing instead on functions, event triggers, and stateless compute. These patterns matter when applications must scale instantly, handle variable workloads, reduce operational overhead, and minimize costs by paying only for execution time. They are ideal for event-driven architectures, real-time data processing, API backends, batch jobs, and microservices with irregular traffic.

Triggering Functions on Events

Functions execute in response to external events. The pattern begins with a producer generating events—messages, file uploads, database changes, or API calls—and a function that consumes them. The key is ensuring the function runs exactly once per event, with minimal latency and no downtime.

An event is typically a JSON object delivered via an HTTP POST or message queue. The function receives it in a standardized format:

{
  "id": "event-123",
  "source": "s3://my-bucket/uploads",
  "type": "file.uploaded",
  "time": "2023-08-15T10:30:00Z",
  "data": {
    "key": "photos/2023/sunset.jpg",
    "size": 428173,
    "bucket": "my-bucket"
  }
}

The function must be idempotent: if the same event is processed twice, the result is identical. A common failure mode is duplicate processing due to message brokers not acknowledging receipt. To prevent this, use a deduplication key derived from the event’s id and a persistent store like a database or key-value store.

For example, a function processing file.uploaded events might:

CREATE TABLE processed_events (
  event_id VARCHAR(255) PRIMARY KEY,
  processed_at TIMESTAMP WITH TIME ZONE
);
INSERT INTO processed_events (event_id, processed_at)
VALUES ('event-123', NOW())
ON CONFLICT (event_id) DO NOTHING;

If the function crashes after inserting the record but before completing processing, the event is not lost, and the next invocation finds the record and avoids reprocessing.

The event delivery mechanism must guarantee at-least-once delivery. A common failure is event loss when the function is cold: it takes 1–3 seconds to start, during which events accumulate. To reduce latency, configure the function to warm up periodically using a scheduled task or keep-alive trigger.

Chaining Functions with Message Queues

When a single function cannot process an event in isolation, chain multiple functions. The output of one function becomes the input of another, forming a pipeline. This pattern scales independently and decouples stages.

Use a message queue to connect functions. The first function publishes messages to a queue and waits for acknowledgment. Subsequent functions consume messages from the queue.

Example: resize image → generate thumbnails → update database.

The queue supports durable, persistent messages with retry policies. A message is considered processed only after the consumer acknowledges it. If the consumer fails before acknowledging, the message is re-delivered.

Messages are structured as JSON with metadata:

{
  "messageId": "msg-987",
  "body": {
    "imageKey": "photos/2023/sunset.jpg",
    "size": "large",
    "format": "webp"
  },
  "attributes": {
    "priority": 5,
    "retries": 3,
    "originalEventId": "event-123"
  }
}

A function consuming from the queue must:

Failure modes include:

Use maxNumberOfMessages to control batch processing. For example, a function processes up to 10 messages at once, reducing per-message overhead.

To avoid backpressure, monitor queue depth and scale function instances accordingly. A function consuming from a queue can be triggered by a cloud provider’s event source (e.g., AWS Lambda, Azure Functions, GCP Cloud Functions), but the pattern is agnostic to the underlying platform.

Stateful Processing with Event Sourcing

When a function must maintain state across multiple events, use event sourcing. Instead of storing the current state in a database, store a sequence of events. The function reconstructs state by replaying all events.

Each event is a record with a type, timestamp, and data. The state is derived by applying events in order.

Example event types:

{
  "type": "user.created",
  "timestamp": "2023-08-15T10:30:00Z",
  "data": {
    "userId": "u-456",
    "name": "Alice Smith",
    "email": "[email protected]"
  }
}
{
  "type": "user.updated",
  "timestamp": "2023-08-15T11:15:30Z",
  "data": {
    "userId": "u-456",
    "name": "Alice Johnson",
    "phone": "+1-555-123-4567"
  }
}

The function reads all events for a user and applies them in chronological order:

SELECT id, type, data, timestamp
FROM user_events
WHERE userId = 'u-456'
ORDER BY timestamp;

Then, apply each event to build the current state:

{
  "userId": "u-456",
  "name": "Alice Johnson",
  "email": "[email protected]",
  "phone": "+1-555-123-4567",
  "lastUpdated": "2023-08-15T11:15:30Z"
}

To avoid recomputing state on every call, maintain a current_state table that caches the result of event replay. On each event write, update the current state.

A common failure is event ordering. If events are written to the database before the function applies them, the state may be inconsistent. Use versioning:

If the function fails during replay, it can restart from the last version. To reduce latency, use a snapshot: store the state at regular intervals (e.g., every 100 events) so replay starts from the latest snapshot.

Building APIs with Serverless Functions

Expose a service via HTTP API using serverless functions. Each function endpoint corresponds to a route. The pattern is simple: HTTP request → function invocation → HTTP response.

The function receives the request as a structured object with:

The response must be a standard HTTP response object with:

Example: GET /users/{id}

{
  "statusCode": 200,
  "headers": {
    "Content-Type": "application/json",
    "Cache-Control": "max-age=3600"
  },
  "body": {
    "id": "u-456",
    "name": "Alice Johnson",
    "email": "[email protected]"
  }
}

The function must handle routing, request validation, and error responses. Use a router to dispatch to handlers based on path and method.

A common failure is cold starts. The function takes 1–2 seconds to initialize. To mitigate, use:

Error handling is critical. A function must respond with appropriate status codes and error details:

{
  "statusCode": 400,
  "body": {
    "error": "InvalidInput",
    "message": "User name is required",
    "details": {
      "field": "name",
      "expected": "string",
      "actual": "null"
    }
  }
}

Use standard HTTP headers for caching, rate limiting, and metadata:

When a function fails to respond, the API gateway or load balancer must handle retries, timeouts, and fallbacks.

Handling Long-Running Workflows

Serverless functions are stateless and short-lived (typically 15 minutes max). For workflows that span hours or days, use a workflow orchestrator.

The pattern involves:

The coordinator maintains a workflow state with:

Each worker function performs one task and updates the workflow state.

Example workflow: process order

  1. order.created → start workflow
  2. validate.order → validate order details
  3. calculate.tax → calculate tax
  4. charge.payment → process payment
  5. ship.order → schedule shipment

The coordinator function is triggered by an event. It:

A worker function for charge.payment:

{
  "taskId": "task-789",
  "workflowId": "wf-123",
  "step": "charge.payment",
  "input": {
    "orderId": "o-567",
    "amount": 249.99,
    "currency": "USD"
  }
}

The worker performs the payment, then updates the workflow state:

UPDATE workflows
SET status = 'completed',
    output = '{"transactionId": "txn-456", "status": "success"}',
    updated = NOW()
WHERE id = 'wf-123';

If a step fails, the workflow can:

A common failure is state inconsistency: a worker updates the workflow but crashes before returning. To prevent this, use a two-phase commit pattern:

The workflow can be resumed from the latest event, even if the coordinator function is down.

Managing Function Configuration and Environment

Serverless functions depend on configuration that varies across environments: development, staging, production.

Configuration is typically stored as key-value pairs or structured JSON. The function reads configuration at startup, and changes are applied without redeployment.

Common configuration keys:

Example config file:

{
  "environment": "production",
  "database": {
    "url": "postgresql://prod-db:5432/ecommerce",
    "poolSize": 10
  },
  "features": {
    "emailNotifications": true,
    "rateLimiting": {
      "maxRequests": 1000,
      "windowSeconds": 60
    }
  },
  "tracing": {
    "enabled": true,
    "sampleRate": 0.1
  }
}

The function loads configuration from:

Use a configuration loader that applies defaults, then overrides with environment-specific values. A common failure is missing environment variables: a function starts but uses default values, leading to incorrect behavior.

To reduce configuration drift, use a declarative approach: define configuration as code, and deploy it with the function.

Use versioned configuration: each deployment uses a specific config version, and changes are tested before promotion. A function can be configured to auto-load the latest config on startup.

Monitoring and Observability

Serverless functions must be observable. The pattern includes logging, metrics, and tracing.

The function writes structured logs using a standard format, such as JSON:

{
  "timestamp": "2023-08-15T11:15:30Z",
  "level": "info",
  "function": "processFile",
  "event": "event-123",
  "message": "Successfully resized image",
  "durationMs": 247,
  "metadata": {
    "inputSize": 428173,
    "outputSize": 183421,
    "resolution": "1920x1080"
  }
}

Metrics are collected automatically: invocation count, duration, memory usage, error rate. The function can emit custom metrics:

{
  "name": "file.processing.duration",
  "value": 247,
  "timestamp": "2023-08-15T11:15:30Z",
  "dimensions": {
    "function": "processFile",
    "size": "large"
  }
}

Tracing connects function invocations across services. A trace ID flows through all services:

A function captures the trace in its logs and headers:

{
  "traceId": "a1b2c3d4-e5f6-7890-g1h2-i3j4k5l6m7n8",
  "spanId": "s123",
  "parentId": "s456"
}

The function propagates the trace through HTTP calls and message queue interactions.

Failure modes include:

Use a monitoring dashboard that correlates logs, metrics, and traces. Alert on:

The pattern is portable: any serverless platform can implement it using standard tools (curl, kubectl, psql, git, systemctl).

This page was rewritten on 16 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.