Federated Learning

Train machine learning models across distributed data sources without centralizing sensitive data. Covers federated averaging, privacy-preserving computation, communication efficiency, heterogeneous data handling, and the patterns that make federated learning practical.

Federated Learning enables training machine learning models across distributed data sources without moving data to a central location. It matters when data privacy regulations prevent data sharing or when network bandwidth limits centralised processing.

Training a Model with Federated Averaging

To train a model using federated averaging, you define a global model and update it iteratively from local clients. The process requires coordination between a central server and multiple edge devices or data silos.

Each client trains a local model using its own dataset and sends model parameters to the server. The server aggregates these updates using a weighted average based on client data size or contribution.

Example Training Flow

Start with a baseline model configuration:

model:
  architecture: "resnet50"
  optimizer:
    type: "sgd"
    lr: 0.01
    momentum: 0.9
  loss: "cross_entropy"

Each client performs local training:

python train_client.py \
  --model-path /tmp/client_model.pth \
  --data-path /local/data \
  --epochs 5 \
  --batch-size 32

The client sends its updated weights to the server:

curl -X POST http://federated-server:8080/submit_weights \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "client_123",
    "weights": "base64_encoded_weights",
    "data_size": 1000
  }'

The server aggregates weights:

curl -X POST http://federated-server:8080/aggregate \
  -H "Content-Type: application/json" \
  -d '{
    "round": 3,
    "client_updates": [
      {"client_id": "client_123", "weights": "base64_encoded_weights", "data_size": 1000},
      {"client_id": "client_456", "weights": "base64_encoded_weights", "data_size": 1500}
    ]
  }'

Configuring Communication Between Clients and Server

Communication relies on HTTP or gRPC endpoints for model updates and status reporting.

Server endpoints expect these request structures:

{
  "client_id": "string",
  "weights": "base64_encoded_string",
  "data_size": 1000,
  "timestamp": "2023-01-01T00:00:00Z"
}

Client-side configuration specifies server address and communication interval:

federated:
  server_url: "http://federated-server:8080"
  upload_interval: 3600
  max_retries: 3

If a client fails to send updates, it retries up to max_retries times before marking itself as offline.

Handling Client Failures

Clients may be temporarily or permanently unavailable. The server tracks client status and adjusts aggregation accordingly.

A client that does not respond within upload_interval seconds is considered inactive. If the same client fails to upload for 3 consecutive rounds, the server removes it from the training process.

Client failure can be detected by:

curl http://federated-server:8080/status \
  -H "Authorization: Bearer token123"

Response:

{
  "round": 5,
  "active_clients": ["client_123", "client_456"],
  "inactive_clients": ["client_789"]
}

The server does not automatically reassign work to inactive clients. A manual intervention is required to resume training for a failed client.

Managing Data Imbalance

When clients have different amounts of data, the server should weight updates appropriately during aggregation.

Each client reports its data size in the update request:

{
  "client_id": "client_123",
  "weights": "base64_encoded_weights",
  "data_size": 1000,
  "timestamp": "2023-01-01T00:00:00Z"
}

The server computes weighted average using this formula:

new_weights = Σ(client_weight * client_weights) / Σ(client_weight)
where client_weight = data_size / total_data_size

If a client sends an update with missing data_size, the server defaults to 1 for that client.

Versioning and Rollback

Model versions are tracked by round number. Each successful aggregation increments the global round counter.

To roll back to a previous version, you can specify a target round:

curl -X POST http://federated-server:8080/rollback \
  -H "Content-Type: application/json" \
  -d '{
    "target_round": 2
  }'

Rollback requires the server to reload model weights from that specific round. If the requested round does not exist, the server returns:

{
  "error": "Round not found",
  "requested_round": 2,
  "available_rounds": [1, 3, 4]
}

Aggregation Strategies

Federated averaging supports several aggregation methods. The server selects one strategy and applies it to all updates.

Example configuration for a robust aggregation:

aggregation:
  method: "weighted_average"
  strategy: "median"
  outlier_threshold: 0.1

Median-based aggregation replaces outliers with median values before averaging:

def median_aggregate(weights_list):
    # Compute medians per parameter
    aggregated = []
    for param_group in weights_list:
        # Use median of all client updates for this group
        aggregated.append(np.median(param_group, axis=0))
    return aggregated

If outlier_threshold is set to 0.1 and a client's update deviates more than 10% from the median, that client's contribution is excluded.

Client Authentication and Authorization

Clients must authenticate before sending updates. The server validates client identity using tokens or certificates.

Example request with authentication:

curl -X POST http://federated-server:8080/submit_weights \
  -H "Authorization: Bearer client_token_abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "client_123",
    "weights": "base64_encoded_weights",
    "data_size": 1000
  }'

If the token is invalid, the server returns:

{
  "error": "Unauthorized",
  "message": "Invalid or expired client token"
}

Clients can refresh their tokens using a dedicated endpoint:

curl -X POST http://federated-server:8080/refresh_token \
  -H "Authorization: Bearer old_token"

Monitoring and Metrics

The server exposes metrics to track training progress:

curl http://federated-server:8080/metrics

Response includes:

{
  "round": 5,
  "clients_connected": 12,
  "clients_active": 10,
  "average_accuracy": 0.87,
  "last_update": "2023-01-01T00:00:00Z"
}

Accuracy metrics are computed on a sample of clients and may not reflect the full dataset.

Error Handling

Server errors return structured JSON responses:

{
  "error": "Invalid request",
  "message": "Missing required field 'weights'"
}

Client-side errors include:

If a client sends malformed weights, the server skips that update and logs an error.

Configuration Examples

Complete server config file:

server:
  port: 8080
  max_clients: 100
  aggregation_interval: 3600
  model_path: "/models/global_model.pth"
  storage_backend: "local"

federated:
  strategy: "weighted_average"
  outlier_threshold: 0.1
  min_clients_per_round: 5

auth:
  token_expiration_hours: 24
  enable_client_auth: true

Client config file:

client:
  server_url: "http://federated-server:8080"
  model_path: "/tmp/client_model.pth"
  upload_interval: 3600
  max_retries: 3

federated:
  client_id: "client_123"
  data_path: "/local/data"

Common Failure Modes

Silent Data Loss

If a client sends an update with invalid weights, the server silently skips it. No error is returned to the client.

Incorrect Aggregation

Using a median strategy without setting outlier_threshold leads to no outlier detection. Updates from clients with very different data distributions can skew the global model.

Client Timeout

Clients may timeout during upload due to network issues or large model sizes. The server does not retry failed uploads automatically. Manual re-upload is required.

Authentication Token Expiry

If a client's token expires during training, subsequent uploads fail until the client refreshes its token manually.

Inconsistent Data Sizes

Clients without reporting data_size contribute equally to aggregation, regardless of their actual data volume. This can bias the global model toward clients with larger datasets.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.