Production-ready guide covering choreography vs orchestration pattern decision guide with implementation patterns, code examples, and anti-patterns for enterprise engineering teams.
This guide helps determine whether to use choreographed or orchestrated communication between services in distributed systems. The choice affects system complexity, error handling, and observability. Use when designing service interactions where components must coordinate without a central authority (choreography) or through a single controlling entity (orchestration).
Choreography works when services publish and subscribe to shared event streams. The interaction is defined by message flow rather than explicit coordination.
# Example: Kafka-based choreographed system
kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: order-service
spec:
selector:
app: order-service
ports:
- port: 8080
---
apiVersion: v1
kind: ConfigMap
metadata:
name: event-config
data:
topics: |
order.created
payment.processed
inventory.reserved
EOF
Services in a choreographed system don't depend on each other's availability or response times.
# Example service definition with event-driven behavior
apiVersion: apps/v1
kind: Deployment
metadata:
name: notification-service
spec:
replicas: 2
selector:
matchLabels:
app: notification-service
template:
metadata:
labels:
app: notification-service
spec:
containers:
- name: service
image: example/notification-service:latest
env:
- name: EVENT_BUS_URL
value: "kafka://broker:9092"
- name: SUBSCRIBE_TOPICS
value: "order.created,payment.processed"
Each bounded context manages its own data and communication through events.
# Example event producer
curl -X POST http://event-bus.local/events \
-H "Content-Type: application/json" \
-d '{
"type": "order.created",
"data": {
"orderId": "12345",
"customerId": "67890"
}
}'
Use orchestration when you must ensure steps execute in a specific sequence or handle conditional flows.
# Example workflow definition
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: order-processing
spec:
entrypoint: process-order
templates:
- name: process-order
steps:
- - name: validate-order
template: validate
- - name: reserve-inventory
template: reserve
- - name: process-payment
template: payment
Orchestration provides a centralized view of execution state.
// Example workflow engine code
type Workflow struct {
ID string
State map[string]interface{}
Steps []Step
}
func (w *Workflow) Execute() error {
for _, step := range w.Steps {
if err := step.Execute(w.State); err != nil {
return err
}
}
return nil
}
Orchestration allows implementing rollback, compensation, or retry logic.
# Example workflow with error handling
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: order-processing
spec:
entrypoint: process-order
onExit: cleanup-workflow
templates:
- name: process-order
steps:
- - name: validate-order
template: validate
onExit: handle-validation-error
# Example consumer setup
kubectl create deployment order-consumer \
--image=example/order-consumer:latest \
--env=EVENT_TOPIC=order.created \
--env=CONSUMER_GROUP=order-processing-group
{
"eventId": "123e4567-e89b-12d3-a456-426614174000",
"eventType": "order.created",
"timestamp": "2023-01-01T00:00:00Z",
"payload": {
"orderId": "12345",
"customerId": "67890",
"items": [
{
"productId": "p123",
"quantity": 2
}
]
}
}
# Example workflow engine config
workflow:
persistence:
driver: postgres
connection:
host: db.example.com
port: 5432
database: workflow_db
execution:
timeout: 300s
retryAttempts: 3
# Example workflow definition using Argo Workflows
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: user-registration
spec:
entrypoint: register-user
templates:
- name: register-user
steps:
- - name: validate-input
template: validate
- - name: create-user
template: create-user
- - name: send-welcome-email
template: email
# Example: Consumer not configured to receive events
kubectl get pods -l app=notification-service
# Output shows no running notification service pods
# Events are published but never consumed
# Example: Message queue with at-least-once delivery
# Consumer processes same message multiple times due to retry logic
# Result: Duplicate order confirmation emails
# Example: Dead letter topic not monitored
kubectl get pods -l app=message-processor
# No error messages visible in logs
# Messages sit in dead letter queue unnoticed
# Example: Database connection issues
workflow-engine logs:
# Error: failed to persist workflow state: connection refused
# Workflow continues without saving progress
# No retry mechanism for state updates
// Example: Incorrect timeout handling
func processOrder() error {
// This timeout affects only one service call
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// But workflow execution might have longer timeouts
// Result: partial success with inconsistent state
}
# Example: Workflow engine restarts mid-execution
kubectl get pods -l app=workflow-engine
# Output shows multiple restarts
# Running workflows lose their state
# Manual recovery required
# Event bus configuration
eventBus:
brokers:
- host: kafka-broker-1
port: 9092
topics:
- name: order.created
partitions: 3
- name: payment.processed
partitions: 3
# Example consumer group configuration
kubectl create configmap consumer-config \
--from-literal=consumer.group.id=order-processing-group \
--from-literal=auto.offset.reset=earliest
# Workflow engine configuration
workflowEngine:
maxRetries: 3
retryDelay: 5s
defaultTimeout: 300s
persistence:
driver: postgresql
connectionString: "postgresql://user:pass@db:5432/workflow"
# Example workflow with execution timeouts
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: order-processing
spec:
entrypoint: process-order
activeDeadlineSeconds: 300
ttlSecondsAfterFinished: 3600
# Example monitoring setup for event processing
kubectl create serviceaccount event-monitoring
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: event-monitor-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: event-monitor-binding
subjects:
- kind: ServiceAccount
name: event-monitoring
roleRef:
kind: Role
name: event-monitor-role
EOF
# Example: Monitoring event consumption time
kubectl logs -l app=event-processor --since=1h | \
grep -E "(event|processing)" | \
awk '{print $1, $2, $3}'
# Example workflow metrics configuration
workflowMetrics:
enabled: true
prometheus:
port: 9090
path: /metrics
metrics:
- name: workflow_duration_seconds
description: Duration of workflow execution
- name: workflow_failures_total
description: Total number of workflow failures
# Example: Workflow status monitoring
kubectl get workflows -o jsonpath='{.items[*].status.phase}'
# Output shows running, succeeded, failed, or unknown states
# Example: Adding workflow engine to existing event system
kubectl create deployment workflow-engine \
--image=example/workflow-engine:latest \
--env=EVENT_BUS_URL=kafka://broker:9092
# Example: Removing workflow engine
kubectl delete deployment workflow-engine
# Existing services continue publishing events
# New consumers handle coordination logic
# Example: Creating topics with proper configuration
kafka-topics.sh --create \
--topic order.created \
--partitions 3 \
--replication-factor 2 \
--bootstrap-server kafka-broker:9092
# Example: Installing Argo Workflows
kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/stable/manifests/install.yaml
# Example: Testing event flow with mock consumers
kubectl run test-consumer \
--image=example/test-consumer:latest \
--env=EVENT_TOPIC=test.event \
--env=TEST_MODE=true
# Example workflow test definition
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: test-workflow
spec:
entrypoint: test-steps
templates:
- name: test-steps
steps:
- - name: test-step-1
template: test-step
# Example: Kafka producer configuration
kafka:
producer:
linger.ms: 5
batch.size: 16384
compression.type: gzip
# Example: Workflow engine resources
workflowEngine:
resources:
limits:
cpu: "1"
memory: 512Mi
requests:
cpu: 500m
memory: 256Mi
The decision between choreography and orchestration depends on your system's coordination requirements. Choreography provides loose coupling and scalability but requires careful event design. Orchestration provides explicit control but introduces complexity in state management. Choose based on your specific needs for coupling, state management, and error handling.
This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy