Production engineering guide for configuration management patterns covering patterns, implementation strategies, and operational best practices.
Configuration management is the practice of defining, storing, and deploying application settings and environment-specific values separately from source code. This approach enables applications to operate correctly across development, staging, and production environments without code changes. Configuration management matters when you need to change behavior or connectivity without rebuilding or redeploying software.
Environment variables are the most common way to externalize configuration. The standard method uses a simple key-value mapping where each setting is exposed as a variable with a known name.
# Example of setting environment variables for an application
export DATABASE_URL="postgresql://user:pass@host:5432/mydb"
export REDIS_URL="redis://localhost:6379"
export LOG_LEVEL="info"When reading configuration in code, use standard language functions to access these variables:
import os
database_url = os.environ.get('DATABASE_URL')
redis_url = os.environ.get('REDIS_URL', 'redis://localhost:6379')
log_level = os.environ.get('LOG_LEVEL', 'info')The main failure mode occurs when a required variable is missing. Applications should validate required configuration at startup and fail fast:
required_vars = ['DATABASE_URL', 'REDIS_URL']
for var in required_vars:
if not os.environ.get(var):
raise RuntimeError(f"Missing required environment variable: {var}")Configuration files store settings in structured formats like JSON, YAML, or TOML. These files are typically loaded once at application startup and then cached.
# config.yaml
database:
host: "localhost"
port: 5432
name: "myapp"
username: "appuser"
password: "secret"
redis:
host: "localhost"
port: 6379
db: 0Applications read this file with standard configuration libraries:
import yaml
with open('config.yaml', 'r') as f:
config = yaml.safe_load(f)
database_url = f"postgresql://{config['database']['username']}:{config['database']['password']}@{config['database']['host']}:{config['database']['port']}/{config['database']['name']}"When configuration files are missing, applications should provide defaults or fail with clear error messages:
try:
with open('config.yaml', 'r') as f:
config = yaml.safe_load(f)
except FileNotFoundError:
raise RuntimeError("Configuration file config.yaml not found")Secrets like API keys, database passwords, and encryption keys should never appear in source code or version-controlled configuration files.
# Example of using a secrets manager to retrieve credentials
SECRET_KEY=$(aws secretsmanager get-secret-value --secret-id myapp/database-password --query SecretString --output text)
export DATABASE_PASSWORD=$SECRET_KEYApplications retrieve secrets at startup using appropriate APIs:
import boto3
def get_secret(secret_name):
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name='us-east-1'
)
get_secret_value_response = client.get_secret_value(
SecretId=secret_name
)
return get_secret_value_response['SecretString']The most common failure with secret management is the accidental exposure of secrets in logs or error messages:
# Bad: logging secrets directly
database_password = get_secret('myapp/database-password')
logger.info(f"Connecting to database with password {database_password}")
# Good: mask secrets in logs
logger.info("Connecting to database with password [REDACTED]")Secrets can also be passed through environment variables, but this requires careful handling of the variable values themselves.
# Setting a secret as an environment variable
export API_KEY="sk-1234567890abcdef"Applications should validate that secrets are present and not empty:
api_key = os.environ.get('API_KEY')
if not api_key:
raise RuntimeError("API_KEY environment variable is required")
if len(api_key) < 10:
raise RuntimeError("API_KEY appears to be invalid or truncated")Applications can watch for configuration changes and reload settings without restarting.
import os
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class ConfigReloadHandler(FileSystemEventHandler):
def on_modified(self, event):
if event.src_path.endswith('.yaml'):
# Reload configuration
reload_config()
def watch_config():
observer = Observer()
observer.schedule(ConfigReloadHandler(), path='.', recursive=False)
observer.start()The main failure mode for dynamic reloading is race conditions where configuration changes during a read operation. Applications should use atomic file operations or locking mechanisms.
When configuration files are updated, maintaining version control helps track changes and rollback when necessary.
# Example of versioned configuration management
git add config.yaml
git commit -m "Update database connection settings"
git tag v1.2.3Applications should verify configuration integrity by checking file hashes or checksums:
import hashlib
def validate_config(config_file):
with open(config_file, 'rb') as f:
file_hash = hashlib.sha256(f.read()).hexdigest()
# Compare against known good hash
expected_hash = "abc123def456..."
if file_hash != expected_hash:
raise RuntimeError("Configuration file has been modified unexpectedly")Different environments require different settings. The standard approach is to use environment-specific files or prefixes.
# Development environment
export ENVIRONMENT="development"
export LOG_LEVEL="debug"
# Production environment
export ENVIRONMENT="production"
export LOG_LEVEL="error"Applications can have conditional logic based on the environment:
environment = os.environ.get('ENVIRONMENT', 'development')
if environment == 'development':
log_level = 'debug'
debug_mode = True
elif environment == 'production':
log_level = 'error'
debug_mode = FalseApplications can merge configuration from multiple sources, with later values overriding earlier ones.
# Load default config
config = {
'database': {
'host': 'localhost',
'port': 5432
}
}
# Override with environment-specific settings
env_config = {
'database': {
'host': os.environ.get('DB_HOST', 'localhost'),
'port': int(os.environ.get('DB_PORT', '5432'))
}
}
# Merge configurations
for key, value in env_config.items():
if isinstance(value, dict):
for sub_key, sub_value in value.items():
config[key][sub_key] = sub_value
else:
config[key] = valueApplications should validate configuration against expected formats to prevent runtime errors.
import jsonschema
config_schema = {
"type": "object",
"properties": {
"database": {
"type": "object",
"properties": {
"host": {"type": "string"},
"port": {"type": "integer", "minimum": 1, "maximum": 65535}
},
"required": ["host", "port"]
}
},
"required": ["database"]
}
def validate_config(config):
try:
jsonschema.validate(config, config_schema)
except jsonschema.ValidationError as e:
raise RuntimeError(f"Configuration validation failed: {e.message}")Applications should check for required configuration values and fail immediately if they are missing.
required_keys = ['DATABASE_URL', 'REDIS_URL']
for key in required_keys:
if not os.environ.get(key):
raise RuntimeError(f"Required configuration key {key} is missing")When configuration management grows organically, it often becomes difficult to track which values are actually used.
# Bad: too many unused variables
export DATABASE_URL="postgresql://..."
export REDIS_URL="redis://..."
export LOG_LEVEL="info"
export API_KEY="secret-key"
export DEBUG_MODE="false"
export MAX_CONNECTIONS="100"
export TIMEOUT_SECONDS="30"
export SSL_ENABLED="true"
export CUSTOM_HEADER="X-Custom-Header"Applications should track which configuration keys are actually consumed:
# Log unused configuration
used_keys = set()
for key in config.keys():
if key in used_keys:
logger.debug(f"Using configuration key: {key}")
else:
logger.warning(f"Unused configuration key: {key}")When configuration is cached, updates may not be immediately visible to running applications.
# Example of configuration caching with expiration
import time
class CachedConfig:
def __init__(self):
self._config = None
self._last_updated = 0
self._cache_ttl = 300 # 5 minutes
def get_config(self):
if not self._config or (time.time() - self._last_updated) > self._cache_ttl:
self._config = load_config_from_source()
self._last_updated = time.time()
return self._configDifferent environments may have different configuration structures, causing runtime issues.
# Development uses local database
export DATABASE_URL="postgresql://localhost/mydb"
# Production uses remote database
export DATABASE_URL="postgresql://prod-db.example.com/mydb"Applications should validate the configuration structure at startup:
def validate_database_config(config):
db_url = config.get('database', {}).get('url')
if not db_url:
raise RuntimeError("Database URL is required")
# Validate database connection string format
if not db_url.startswith(('postgresql://', 'mysql://')):
raise RuntimeError("Database URL must use supported protocol")The most critical pattern for successful configuration management is to treat configuration as immutable data that can be validated, versioned, and audited. Applications should fail fast when configuration is invalid, and should never silently accept incorrect settings that cause runtime failures.
This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.
We use cookies for analytics (Google Analytics) and advertising (Google AdSense) to improve your experience and support free content. Privacy Policy