Production engineering guide for feature flag engineering covering patterns, implementation strategies, and operational best practices.
Feature flag engineering is the disciplined practice of managing conditional code paths in software through declarative, runtime-controlled toggles. It matters when teams need to decouple feature releases from code deploys—especially in continuous delivery environments where multiple features are in flight simultaneously, and rollbacks must be fast, safe, and observable. Feature flags enable gradual rollouts, A/B testing, dark launches, and canary deployments without redeploying services.
A feature flag is a boolean or structured value stored in a centralized configuration system and queried at runtime. The flag’s state controls whether a code path executes. Flags are typically named with a consistent prefix: user.profile.legacy-ui, checkout.enable.stripe.payments, analytics.enable.realtime.metrics.
Configuration is distributed via a dedicated service or stored in a database. A typical flag entry might be:
{
"id": "checkout.enable.stripe.payments",
"name": "Enable Stripe Payments in Checkout",
"type": "boolean",
"default": true,
"targets": [
{
"type": "user",
"values": ["[email protected]", "[email protected]"],
"variant": "beta"
},
{
"type": "segment",
"name": "premium_customers",
"variant": "alpha"
}
],
"variants": {
"alpha": {
"payment_processor": "stripe",
"timeout_ms": 2000
},
"beta": {
"payment_processor": "stripe",
"timeout_ms": 1500,
"enable_recurring": true
}
}
}The flag’s default value is used when no specific targeting rule matches. targets define which users or groups get a variant, and variants define the configuration associated with a named variant.
At runtime, a feature flag evaluation library checks if a flag is active for a given context. A common pattern uses a FeatureFlag interface in a service:
type FeatureFlag interface {
IsActive(ctx context.Context, name string, context map[string]interface{}) (bool, error)
GetVariant(ctx context.Context, name string, context map[string]interface{}) (map[string]interface{}, error)
}
func EnableStripePayments(ctx context.Context, user User) (bool, error) {
flag, err := featureFlag.IsActive(ctx, "checkout.enable.stripe.payments", map[string]interface{}{
"user_id": user.ID,
"country": user.Country,
"tier": user.Tier,
})
if err != nil {
return false, err
}
return flag, nil
}Evaluation follows a prioritized order:
user_id, email, session_id, etc.segment, region, device_type, etc.default value.When evaluating, a service might return a 500 error if the flag service is unreachable:
{
"error": "FeatureFlagServiceUnavailable",
"message": "Unable to reach feature flag configuration store",
"details": {
"url": "https://flags.example.com/api/v1/flags",
"timeout_ms": 1000
},
"status": 503
}The client library should cache flag values locally to avoid network latency and handle transient failures. A typical cache key is flag:<name>:<context_hash>, where context_hash is a SHA-256 digest of the context map.
To enable a feature in production, a developer creates a new flag and sets it as default: true in the configuration store. The flag is then activated via a simple HTTP request:
curl -X PUT https://flags.example.com/api/v1/flags/checkout.enable.stripe.payments \
-H "Content-Type: application/json" \
-d '{
"default": true,
"description": "Enable Stripe as primary payment processor in checkout"
}'The service responds with:
{
"id": "checkout.enable.stripe.payments",
"status": "updated",
"last_updated": "2024-08-25T10:00:00Z"
}This triggers an immediate rollout: all new users and services that evaluate the flag now see the new behavior.
A gradual rollout uses targeting to progressively expose a feature. For example, a team wants to roll out a new checkout UI in three stages:
The process begins by adding a percentage target:
{
"type": "percentage",
"value": 10,
"variant": "beta"
}The evaluation logic uses a random seed derived from the user_id, ensuring consistent assignment. A user_id of 12345 gets a consistent 10% probability over time.
When the team wants to expand to premium users, they add a segment:
{
"type": "segment",
"name": "premium_customers",
"variant": "alpha"
}The segment premium_customers is defined as users whose tier is premium or enterprise. The evaluation logic checks for segment membership by querying a segments table:
SELECT user_id, created_at
FROM user_segments
WHERE segment_name = 'premium_customers'
AND created_at > '2024-08-01';If a user is in multiple segments or multiple targets, the first match wins. If no match, the default value applies.
Flags support richer configurations beyond simple booleans. A variant can be used to enable multiple feature behaviors. For example:
{
"name": "analytics.enable.realtime.metrics",
"type": "variant",
"default": "basic",
"variants": {
"basic": {
"polling_interval_ms": 30000,
"aggregation": "count"
},
"advanced": {
"polling_interval_ms": 5000,
"aggregation": "sum",
"enable_streaming": true,
"window_size": "1h"
}
}
}A service uses the variant to configure a real-time pipeline:
func startAnalyticsPipeline(ctx context.Context, flag FeatureFlag) error {
variant, err := flag.GetVariant(ctx, "analytics.enable.realtime.metrics", map[string]interface{}{
"user_id": 12345,
"region": "us-west",
})
if err != nil {
return err
}
pollingInterval, ok := variant["polling_interval_ms"].(int)
if !ok {
pollingInterval = 30000
}
enableStreaming, ok := variant["enable_streaming"].(bool)
if !ok {
enableStreaming = false
}
// Configure pipeline with these values
return startStream(pollingInterval, enableStreaming)
}The variant is evaluated and cached per context, allowing multiple configurations to coexist.
A critical failure mode is a mismatch between flag configuration and code behavior. A flag might be enabled in the config store, but the code expects a different structure or field name. The system must validate flag definitions.
A validation endpoint checks if all required fields are present and correct:
curl -X POST https://flags.example.com/api/v1/flags/validate \
-H "Content-Type: application/json" \
-d '{
"flag_name": "checkout.enable.stripe.payments",
"context": {
"user_id": 12345,
"country": "US"
},
"expected": {
"type": "boolean",
"required_fields": ["payment_processor", "timeout_ms"]
}
}'The response includes:
{
"valid": true,
"issues": [],
"evaluated": {
"flag": "checkout.enable.stripe.payments",
"value": true,
"variant": "beta",
"evaluation_time_ms": 42
}
}If validation fails, it returns a list of issues:
{
"valid": false,
"issues": [
{
"type": "missing_field",
"field": "payment_processor",
"expected_type": "string",
"actual_value": null
},
{
"type": "invalid_type",
"field": "timeout_ms",
"expected_type": "integer",
"actual_value": "3000"
}
]
}This allows teams to catch configuration drift early and automate flag validation in CI.
A common failure is a flag being enabled but not active due to a mismatch in context keys. The service expects user_id, but the flag is keyed by userId (camelCase). The evaluation returns false even when the flag should be true.
A second failure mode is stale flag caches. If a flag is updated in the config store but clients use a cached value, the rollout appears delayed. The cache should be invalidated on every flag update via a Cache-Control: max-age=300 header and a ETag header:
GET /api/v1/flags/checkout.enable.stripe.payments HTTP/1.1
Host: flags.example.com
If-None-Match: "abc123"If the ETag matches, the server returns 304 Not Modified. Otherwise, it returns the full flag and updates the ETag.
When multiple targets apply to the same user, the evaluation order determines which variant takes precedence. If a user is in two segments, and both have different variants, the first match wins. But if the service expects the most specific rule to apply, it must explicitly define the priority order.
A subtle issue arises when a flag has a percentage target and a user target that overlap. A user might be in both the 10% sample and manually targeted. If the system applies both, the user might receive a variant that combines both configurations, leading to inconsistent behavior.
To avoid this, the evaluation logic should support a merge_strategy option:
{
"type": "percentage",
"value": 10,
"variant": "beta",
"merge_strategy": "override"
}With merge_strategy: "override", the user target overrides the percentage target. With merge_strategy: "merge", the values are combined.
As features evolve, flags become part of a larger system. Teams should version their flags. A flag can have a version field:
{
"name": "checkout.enable.stripe.payments",
"version": "2.1",
"description": "Enable Stripe with improved UX and fallback",
"deprecated": true,
"deprecation_reason": "Use checkout.enable.stripe.v2 instead"
}A GET /api/v1/flags endpoint returns a list of all flags, including version, deprecation status, and change log.
The system should emit events when flags are updated:
{
"event_type": "flag_updated",
"timestamp": "2024-08-25T10:05:33Z",
"flag_name": "checkout.enable.stripe.payments",
"previous_value": true,
"new_value": false,
"updated_by": "[email protected]",
"metadata": {
"commit": "abc123def456",
"source": "ci/production"
}
}These events are consumed by dashboards, alerting systems, and observability tools.
Feature flag engineering is the art of making code dynamic through declarative, versioned, and targetable configuration. It enables teams to release features independently, roll them out incrementally, and validate behavior in production without redeploying. The core mechanics—flag lifecycle, context-based evaluation, targeting, variants, and validation—form a predictable, observable, and robust system. The key to success lies in consistency: using a single, well-documented format for flags, enforcing validation, and designing for common failure modes. When a flag is misconfigured, the system must not fail silently but instead provide clear signals and actionable diagnostics.
This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy