Schema Registry Management | The Garnet Wiki

Schema registry management handles versioning, validation, and evolution of data schemas used in streaming and batch processing pipelines. This matters when systems need to maintain compatibility across producers and consumers while allowing controlled schema changes.

Registering New Schemas

To register a new schema, POST to the /subjects/{subject}/versions endpoint with the schema content:

POST /subjects/my-topic/versions
Content-Type: application/vnd.schemaregistry.v1+json

{
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"}]}"
}

The response includes the assigned version number:

{
  "id": 123,
  "version": 1,
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"}]}"
}

Schema Evolution Rules

Schemas evolve according to strict compatibility rules. When registering a new version, use the compatibility parameter:

POST /subjects/my-topic/versions
Content-Type: application/vnd.schemaregistry.v1+json
Accept: application/vnd.schemaregistry.v1+json

{
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"email\",\"type\":\"string\"}]}"
}

The system checks compatibility against the latest registered version. If backward compatibility is configured, adding optional fields passes validation.

Schema Validation During Registration

Validation occurs during registration. The registry may reject schemas with invalid JSON or incompatible field types:

POST /subjects/my-topic/versions
Content-Type: application/vnd.schemaregistry.v1+json

{
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"}]}"
}

If validation fails, the server returns HTTP 400 with error details:

HTTP/1.1 400 Bad Request
Content-Type: application/vnd.schemaregistry.v1+json

{
  "error_code": 400,
  "message": "Invalid schema: field name 'name' cannot be null"
}

Version Management

List all versions of a subject:

GET /subjects/my-topic/versions
Accept: application/vnd.schemaregistry.v1+json

Response contains version numbers:

[1, 2, 3]

Retrieve a specific version:

GET /subjects/my-topic/versions/2
Accept: application/vnd.schemaregistry.v1+json

Response includes the full schema definition and metadata:

{
  "subject": "my-topic",
  "version": 2,
  "id": 123,
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"}]}"
}

Schema Compatibility Configuration

Set compatibility level for a subject:

PUT /subjects/my-topic/config
Content-Type: application/vnd.schemaregistry.v1+json

{
  "compatibility": "BACKWARD"
}

Available compatibility levels include:

NONE: No compatibility checks
BACKWARD: New schema must be backward compatible
FORWARD: New schema must be forward compatible
FULL: New schema must be both backward and forward compatible

Schema Deletion

Delete a specific version:

DELETE /subjects/my-topic/versions/2

The registry returns 200 OK on successful deletion. Deleting the last version removes the subject entirely.

Schema Lookup by ID

Find schema by its globally unique identifier:

GET /schemas/ids/123
Accept: application/vnd.schemaregistry.v1+json

Response includes schema content and subject information:

{
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"}]}"
}

Schema Evolution Testing

Test compatibility before registration using the validate endpoint:

POST /compatibility/subjects/my-topic/versions/latest
Content-Type: application/vnd.schemaregistry.v1+json

{
  "schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"email\",\"type\":\"string\"}]}"
}

Response indicates compatibility status:

{
  "is_compatible": true
}

Error Handling

The registry returns specific error codes for schema registration issues. Common errors include:

400: Invalid schema format
409: Incompatible schema version
422: Schema validation failed
500: Internal server error during schema processing

Configuration Options

Registry behavior is controlled through these configuration parameters:

compatibility.level: Set default compatibility for new subjects
schema.cache.size: Configure internal schema cache capacity
leader.eligibility: Enable/disable leader election for high availability

Example configuration file fragment:

schema:
  cache:
    size: 1000
  compatibility:
    level: BACKWARD

Schema Migration Strategy

For migration, register new schemas with backward compatibility. Use the same subject name but different version numbers. Consumers can then gradually switch to newer versions without breaking existing functionality.

Handling Schema Conflicts

When multiple producers write to the same topic, ensure all use compatible schema versions. The registry enforces compatibility rules at registration time. Producers failing to register schemas correctly may cause runtime failures downstream.

Schema Registry Health Checks

Verify registry availability with:

GET /subjects

Response lists registered subjects. A healthy registry returns 200 OK with an array of subject names.

Schema Version Rollback

If a newer version introduces issues, rollback to previous version by re-registering that schema. The registry maintains all versions and allows re-registration of previously deleted versions.

Schema Evolution Failure Modes

Failure modes include silent compatibility violations that only surface during consumption. A producer registering a schema with BACKWARD compatibility that removes required fields may cause consumers to fail at runtime when processing records.

Configuration Pitfalls

Using NONE compatibility level without proper validation can lead to downstream failures. The registry validates schema format but does not validate semantic correctness of field types or logical constraints.

Schema Registry Monitoring

Monitor schema registry health through:

GET /admin/metrics

Response includes performance metrics and registry status information.

Schema Version Cleanup

Remove unused schema versions using:

DELETE /subjects/my-topic/versions/1

This removes only the specific version, not the subject itself.

This page was rewritten on 16 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.