Migrating from Axual Connect to Kafka Connect

This guide explains what changes when a connector moves from the shared Axual Connect component to a per-tenant Kafka Connect cluster during the deprecation window. It covers why the move matters, and what happens to connector state once a connector runs on the new cluster. No commands or procedures appear here.

The migration path from Axual Connect to Kafka Connect is still being defined. The sections below describe the parts that are settled and mark the parts that are not. Expect the guidance here to change as the migration procedure is finalised.

Why migration matters now

Axual Connect and Kafka Connect run side by side during a deprecation window, so migration is not optional in the long run. See Why Kafka Connect replaces Axual Connect for the deprecation timeline and the reasons Kafka Connect replaces the shared component. Move every connector off the shared Axual Connect component to a per-tenant Kafka Connect cluster before Axual Connect’s maintenance window closes with the 2027.2 release. Deploy the target cluster first with How to Deploy a Kafka Connect Cluster, which lists the broker and Vault prerequisites it depends on.

Migration is not a config-only change, because connector state does not travel with the connector definition. A connector’s progress through its data lives in Kafka or in a Kafka Connect internal topic, not in the connector configuration, so moving the definition to a new cluster leaves that progress behind unless it is carried across deliberately. The two connector types track that progress differently, which is why sink and source connectors need different handling.

What happens to sink connector state

Sink connectors read from Kafka topics and write to an external system, and their progress is a Kafka consumer group offset. A sink connector belongs to a Kafka consumer group, and Kafka records how far that group has read in each topic partition as a committed offset. That offset is the connector’s only record of what it has already delivered to the external system. This is the connector’s own consumer group, not the worker groupId the Kafka Connect chart derives for coordinating its workers.

Preserving the consumer group ID is what keeps a migrated sink connector reading from where it left off. When a sink connector moves to a Kafka Connect cluster with the same consumer group ID, Kafka hands it the group’s committed offsets and the connector resumes from the next unread record. Change or drop the group ID and Kafka treats the connector as a brand-new consumer, which then follows the group’s auto.offset.reset policy:

  • With a reset policy of earliest, the connector re-reads the topic from the start and re-delivers every record the old connector already wrote, producing duplicates in the external system.

  • With a reset policy of latest, the connector starts at the current end of the topic and skips every record written between the old connector stopping and the new one starting, producing a gap in the external system.

Both outcomes are silent: neither the connector nor Kafka reports an error, because a new consumer group with no committed offsets is a valid state. Preserving the group ID is the settled part of sink connector migration.

What happens to source connector state

Source connectors read from an external system and write to Kafka topics, and their progress is a source offset rather than a Kafka consumer offset. A source offset is a connector-defined marker of how far it has read in the external system, such as a file position, a database row identifier, or a log sequence number. Kafka Connect stores these markers in its own offset storage topic, one of the three internal topics each cluster keeps, described in the concepts page.

Because each Kafka Connect cluster owns a separate offset storage topic, a source connector’s markers do not exist on the new cluster until they are carried across. A source connector that starts on a new cluster with no stored offset begins reading the external system from wherever its own default dictates, which can re-emit records the old connector already wrote to Kafka. Carrying the offsets across so the connector resumes cleanly may require operator involvement, because the markers sit in an internal topic rather than in the connector configuration.

The exact source-connector offset-migration procedure is not yet defined. Do not treat any specific method for copying source offsets between clusters as settled until a documented procedure exists. Until then, plan a source connector migration on the assumption that offsets need deliberate handling and that a naive move can re-emit already-delivered records.