Distributor 5.0.3 Changelog

All notable changes to this project will be documented in this file.

5.3.4 2024-11-11

Fix OffsetCommitter issue that target fields are validated but provided to client code with the wrong values, causes the committer deployment to fail

Upgrade dependencies to fix reported CVEs
Upgrade base images
Introduce released image tags based on Strimzi operator version for compatibility
Upgraded expose rules for JMX metrics
Use client ids with the connector name and task for internal Kafka communication.
Added initial dashboards to project

Fix issue with Helm Charts that default patterns values were not set in connector configuration

Implement Offset Distributor and Offset Committer to synchronize offsets between clusters

Create a separate helm chart for Kafka Connect Distributor/Distributions
Replace queue based Kafka Synchronisation Connectors with new implementation for Message Distributor
Implement backpressure
Add explicit distribution metrics
Partition Exception metric was updated only as a produce exception, ignoring other exceptions like topic resolve exceptions
Upgrade to Kafka 3.4.0
Upgrade to Java 11
Upgrade to Strimzi Kafka Connect container 0.35.1

Upgrade clients to Kafka version 3.2.3
Upgrade kafka-synchronization-connectors to version 1.0.3
For message distributor and schema distributor, ClientId is composed by “tenant-instance” (before it was “instance”)
Upgrade Strimzi to version 0.31.1 with Kafka 3.2.3
Upgrade Axual Client to version 6.0.4 with Kafka 3.2.3

Added support for system topics (_tenant-instance-metrics-provider and _tenant-instance-connect.*) to be distributed without topic resolving

Added remote producer clientId to message distributor. Format messageDistributor-$insance-$taskId
Moved Schema Distributor to separate module
Added remote producer clientId to schema distributor. Format schemaDistributor-$insance-$taskId

First implementation using Kafka Connect Principals
Added support to retry for discovery api if it’s unavailable on startup.
Dockerfile now based on axual/strimzi/kafka
Negative and corner cases ITs:
- Topic Deletion/Recreation
- Network connectivity issue between connector producer and remote cluster
Added support to retry for discovery api if it’s unavailable on startup
Removed: Offset Distribution
Supported CPU architectures: linux/amd64,linux/arm64

* Distribute unresolved _<tenant>-<instance>-distribution-monitor topic