Schema Data Evolution

All asynchronous APIs SHOULD leverage Schema Registry to ensure consistency across consumers/producers with regards to message structure and ensuring compatibility across different versions.

The default compatibility mode SHOULD be FULL_TRANSITIVE, which is the default compatibility mode in adidas for Schema Registry. Check the sections below to know more about compatibility modes.

Compatibility modes

Once a given schema is defined, it is unavoidable that the schema evolves with time. Every time this happens, downstream consumers need to be able to handle data with both old and new schemas seamlessly.

Each new schema version is validated according to the configuration before being created as a new version. Namely, it is checked against the configured compatibility types.

Important The mere fact of enabling Schema Registry is not enough to ensure that there are no compatibility issues in a given integration. The right compatibility mode needs also to be selected and enforced.

As a summary, the available compatibility types are listed below:

Mode	Description
BACKWARD	new schema versions are backward compatible with older versions
BACKWARD_TRANSITIVE	backward compatibility across all schema versions, not just the latest one.
FORWARD	new schema versions are compatible with older consumer versions
FORWARD_TRANSITIVE	forward compatibility across all schema versions.
FULL	both backward and forward compatibility with the latest schema version
FULL_TRANSITIVE	both backward and forward compatibility with all schema versions
NONE	schema compatibility checks are disabled

Upgrading process of clients based on compatibility

Depending on the compatibility mode, the process of upgrading producers/consumers will be different based on the compatibility mode enabled.

• NONE

  • As there are no compatibility checks, no order will grant a smooth transition

  • In most of the cases this lead to having to create a new topic for this evolution

• BACKWARD / BACKWARD_TRANSITIVE

  • Consumers MUST be upgraded first before producing new data

  • No forward compatibility, meaning that there's no guarantee that the consumers with older schemas are going to be able to read data produced with a new version

• FORWARD / FORWARD_TRANSITIVE

  • Producers MUST be upgraded first and then after ensuring that no older data is present, upgrade the consumers

  • No backward compatibility, meaning that there's no guarantee that the consumers with newer schemas are going to be able to read data produced with an older version

• FULL / FULL TRANSITIVE

  • No restrictions on the order, anything will work

How to deal with breaking changes

If for any reason you need to use a less strict compatibility mode in a topic, or you can't avoid breaking changes in a given situation, the compatibility mode SHOULD NOT be modified on the same topic.

Instead, a new topic SHOULD be used to avoid unexpected behaviors or broken integrations. This allows a smooth transitioning from clients to the definitive topic, and once all clients are migrated the original one can be decommissioned.

Alternatively, instead of modifying existing fields it MAY be considered as an sub-optimal approach to add the changes in new fields and have both coexisting. Take into account that this pollutes your topic and it can cause some confusion.