Feature Flags & Progressive Delivery

Feature flags and progressive delivery enable teams to release features incrementally, control rollouts with precision, and reduce risk by decoupling code deployment from feature activation. This approach matters when teams deploy frequently, need to validate features with real users before full rollout, or must maintain stability during high-traffic periods—common in cloud-native, microservices-driven systems where releases happen daily, if not hourly.

Activating a Feature Flag in Production

To activate a feature flag in production, developers define a flag in a centralized store, then use it in their application code to gate behavior. The flag is evaluated at runtime, allowing immediate changes without redeploying code.

A typical feature flag is a named boolean toggle stored in a database or key-value service. The application reads the flag value when processing a request. For example, in a Go service using a shared config store:

type FeatureFlags struct {
    EnableNewCheckout bool `json:"enable_new_checkout"`
    UseNewSearchAPI  bool `json:"use_new_search_api"`
    BetaFeatures     bool `json:"beta_features"`
}

// LoadFeatureFlags fetches current flag values from a shared store
func LoadFeatureFlags(ctx context.Context, client *http.Client) (*FeatureFlags, error) {
    req, err := http.NewRequestWithContext(ctx, "GET", "http://config-service:8080/flags", nil)
    if err != nil {
        return nil, err
    }
    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return nil, fmt.Errorf("config service returned %d", resp.StatusCode)
    }

    var flags FeatureFlags
    if err := json.NewDecoder(resp.Body).Decode(&flags); err != nil {
        return nil, err
    }
    return &flags, nil
}

The feature flag is then used in a route:

func HandleCheckout(w http.ResponseWriter, r *http.Request) {
    flags, err := LoadFeatureFlags(r.Context(), client)
    if err != nil {
        http.Error(w, "Failed to load feature flags", http.StatusInternalServerError)
        return
    }

    if flags.EnableNewCheckout {
        // Use new checkout flow
        w.Write([]byte("Rendering new checkout UI"))
    } else {
        // Fall back to old flow
        w.Write([]byte("Rendering legacy checkout UI"))
    }
}

The config service returns a JSON payload:

{
  "enable_new_checkout": true,
  "use_new_search_api": false,
  "beta_features": true
}

When the service is deployed, it reads the flag values from the config service on startup and re-fetches them periodically. The flag value can be updated via API or dashboard without touching the application code.

Managing Flag State Across Environments

Feature flags are not static; their state evolves across environments. To manage this, teams define a hierarchy of flag values: global defaults, environment-specific overrides, and user- or group-specific targeting.

A flag configuration is typically stored in a database table with these columns:

id	name	global_value	environment	user_id	group	created_at	updated_at
1	enable_new_checkout	true	production	NULL	NULL	2023-05-10	2023-05-15
2	enable_new_checkout	false	staging	NULL	NULL	2023-05-10	2023-05-10

The application resolves the flag value by querying the database with a priority order:

User-level override (user_id)
Group-level override (group)
Environment-specific value (environment)
Global default (no environment, user, or group)

The query to resolve a flag for a user in production is:

SELECT COALESCE(
    (SELECT global_value FROM flags WHERE name = $1 AND environment = $2 AND user_id = $3),
    (SELECT global_value FROM flags WHERE name = $1 AND environment = $2 AND group = $3),
    (SELECT global_value FROM flags WHERE name = $1 AND environment = $2),
    (SELECT global_value FROM flags WHERE name = $1)
) AS value
FROM flags
WHERE name = $1;

This allows precise control: for example, a team can enable a feature for all users in the "engineering" group in staging, while keeping it off in production.

Rolling Out Features with Progressive Delivery

Progressive delivery extends feature flags to orchestrate the rollout of a feature across multiple stages.

Gradual Release with Percentage-Based Rollouts

A common pattern is to roll out a feature to a percentage of users. The rollout is managed by a service that evaluates a rollout rule per request.

Each rollout rule is defined in a database table:

id	feature_name	rollout_type	percentage	condition	created_at
1	enable_new_search	percentage	10	null	2023-05-15

The application computes a rollout decision using a hash of the user ID and the feature name:

func IsFeatureEnabled(featureName string, userID string) (bool, error) {
    // Query the rollout rule
    var rule struct {
        FeatureName  string
        RolloutType  string
        Percentage   int
        Condition    string
    }

    err := db.QueryRow(`
        SELECT feature_name, rollout_type, percentage, condition
        FROM rollout_rules
        WHERE feature_name = $1 AND rollout_type = 'percentage'
        ORDER BY created_at DESC
        LIMIT 1
    `, featureName).Scan(&rule.FeatureName, &rule.RolloutType, &rule.Percentage, &rule.Condition)

    if err != nil {
        return false, err
    }

    // Compute hash of user ID
    h := fnv.New32a()
    h.Write([]byte(userID))
    hash := h.Sum32()

    // Check if user falls in the percentage bucket
    threshold := uint32(rule.Percentage * 10000 / 100) // 10% → 10000
    return hash < threshold, nil
}

This method ensures consistent assignment: a user with ID "[email protected]" always sees the feature if the 10% rollout is active.

Targeting Specific User Groups

For more granular control, progressive delivery supports targeting users based on attributes like email domain, role, or region.

A targeting rule is stored in a table:

id	feature_name	target_type	value	condition	created_at
1	enable_new_search	email_domain	example.com	null	2023-05-15

The application evaluates targeting rules in order, returning the first match:

func GetTargetingRule(featureName string, user map[string]string) (*TargetingRule, error) {
    query := `
        SELECT feature_name, target_type, value, condition, created_at
        FROM targeting_rules
        WHERE feature_name = $1
        AND (
            (target_type = 'email_domain' AND $2 LIKE CONCAT('%', value, '%'))
            OR (target_type = 'role' AND $3 = value)
            OR (target_type = 'region' AND $4 = value)
        )
        ORDER BY priority ASC
        LIMIT 1
    `

    var rule TargetingRule
    err := db.QueryRow(query, featureName, user["email"], user["role"], user["region"]).Scan(
        &rule.FeatureName, &rule.TargetType, &rule.Value, &rule.Condition, &rule.CreatedAt,
    )
    if err != nil {
        return nil, err
    }

    return &rule, nil
}

The condition field can contain a JSON expression:

{
  "operator": "and",
  "operands": [
    { "field": "role", "operator": "in", "values": ["admin", "developer"] },
    { "field": "country", "operator": "eq", "value": "US" }
  ]
}

This enables complex targeting: a feature might be enabled for all admins in the US and for users whose email ends in .org.

Ensuring Flag Reliability and Observability

Feature flags are only effective if they are reliable and observable.

Flag Evaluation Failures

When the config service is down, the application must handle flag evaluation failures gracefully. The most common issue is the 503 Service Unavailable from the config service:

Error: config service returned 503
Cause: Network failure, config service crash, database timeout
Solution: Fall back to cached flag values, or use a default set in code.

The application should cache flag values for at least 30 seconds:

type FlagCache struct {
    flags map[string]bool
    mu    sync.RWMutex
    ttl   time.Duration
}

func (c *FlagCache) Get(name string) (bool, bool) {
    c.mu.RLock()
    value, found := c.flags[name]
    c.mu.RUnlock()

    if found {
        return value, true
    }

    return false, false
}

func (c *FlagCache) Set(name string, value bool) {
    c.mu.Lock()
    c.flags[name] = value
    c.mu.Unlock()
}

func (c *FlagCache) LoadFromConfigService(client *http.Client) error {
    resp, err := client.Get("http://config-service:8080/flags")
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("config service returned %d", resp.StatusCode)
    }

    var flags map[string]bool
    if err := json.NewDecoder(resp.Body).Decode(&flags); err != nil {
        return err
    }

    c.mu.Lock()
    c.flags = flags
    c.mu.Unlock()

    return nil
}

If the config service is unavailable, the application continues using the cached values and retries every 30 seconds.

Monitoring and Debugging

The most common debugging issue is a flag that appears enabled in the dashboard but not in the application. This usually stems from a mismatch in environment or user scope.

To debug, the application logs flag resolution details:

{
  "request_id": "abc123",
  "user_id": "[email protected]",
  "feature": "enable_new_search",
  "resolved_value": true,
  "evaluation_path": [
    {
      "source": "user",
      "user_id": "[email protected]",
      "value": true
    },
    {
      "source": "group",
      "group": "engineering",
      "value": false
    },
    {
      "source": "environment",
      "environment": "staging",
      "value": true
    },
    {
      "source": "global",
      "value": false
    }
  ],
  "timestamp": "2023-05-15T10:30:00Z"
}

This log entry reveals that the user has a specific override, but the group override is not active, or that the environment is misconfigured.

Common Pitfalls

Flag not updated in time: A flag is updated in the dashboard, but the service does not refresh its cache for 30 minutes. The change is delayed, leading to confusion.
Misunderstanding targeting scope: A user is in the "engineering" group but the group-specific flag is not applied because the group is named "Engineering" in the dashboard but "engineering" in the database.
Rollout percentage not uniform: The percentage rollout relies on consistent hashing. If the hash function changes (e.g., from fnv32 to murmur3), users shift into or out of the rollout unexpectedly.
Missing fallback logic: When the config service is down, the application uses the default values, but the defaults are outdated or incorrect.

Automating Rollouts with CI/CD

Progressive delivery integrates naturally with CI/CD pipelines. A typical CI/CD job to promote a feature flag includes:

- name: Deploy and Enable Feature Flag
  run: |
    # 1. Deploy new version of service
    kubectl apply -f deploy.yaml

    # 2. Update feature flag via API
    curl -X POST http://config-service:8080/flags \
      -H "Content-Type: application/json" \
      -d '{
        "name": "enable_new_search",
        "environment": "staging",
        "value": true,
        "reason": "Deployed v2.3.0 with improved search performance"
      }'

    # 3. Wait for service to become ready
    kubectl rollout status deployment/search-service

    # 4. Verify flag is active
    while ! curl -f http://config-service:8080/flags?name=enable_new_search; do
      echo "Waiting for flag to be applied..."
      sleep 5
    done

This script ensures that the feature is not only deployed but also activated and validated.

Feature flags and progressive delivery are not a tool but a pattern: a way to manage change in production through dynamic, observable, and composable control points.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.