Kafka Connect: Concepts and Architecture

The sections below cover each area of the component:

The classic Axual Connect component was a single shared Kafka Connect cluster used by all tenants on an instance. That model creates a noisy-neighbour risk: a misbehaving connector on one tenant affects the connector throughput and stability of every other tenant.

Kafka Connect gives each tenant its own Strimzi-managed KC cluster. Each cluster is isolated with its own worker pods, its own internal topics, and its own connector credentials. Tenants choose which cluster to use, and Operators size each cluster independently.

Axual Connect follows a phased deprecation rather than a single cutover release. 2027.2 is the last Axual Connect release with bug fixes and features. After 2027.2, Axual Connect gets critical security fixes only. 2028.2 removes Axual Connect.

Four roles interact with a Kafka Connect cluster, each owning a distinct part of its lifecycle.

Standing up a Kafka Connect cluster is not a single install. The operator produces a set of pieces that together let the Tenant Admin register the cluster and let App Owners run connectors on it. Seeing the whole set first explains why the how-tos are ordered the way they are and why each one depends on the last.

Per cluster, the operator delivers:

The how-tos follow the order these pieces depend on each other: prepare the broker, then set up the Connector Vault, then deploy the cluster, then enable log reading on the Runtime Provisioner, then hand off to the Tenant Admin, then verify. The broker comes first because the worker cannot authenticate without its Access Control Lists (ACLs); the Vault comes before the deploy because the worker resolves credentials at startup; verification comes last because it confirms every earlier piece before the cluster leaves the operator’s hands.

The kafka-connect-helm chart creates a KafkaConnect Custom Resource (CR). The Strimzi operator owns the pod lifecycle from that point. It creates a StrimziPodSet, not a Deployment.

This has one practical consequence for observability: readiness is tracked on the KafkaConnect CR’s Ready condition, not through a Deployment rollout. Tooling that polls Deployment rollout status finds nothing for a Kafka Connect cluster. How to Deploy a Kafka Connect Cluster and Troubleshoot a Kafka Connect Deployment give the exact readiness command.

Strimzi derives the REST API service name from the KafkaConnect CR name. If the CR is renamed because fullnameOverride is not set and the Helm release name changes, Strimzi creates a new service and orphans the old one. Setting fullnameOverride in the chart pins the CR name and prevents this.

Kafka Connect plugins (connectors and Single Message Transforms (SMTs)) must be on the JVM classpath of every worker. The chart supports two delivery mechanisms:

Platform Manager validates that the kafka-topic-name-transforms SMT is present when a Tenant Admin registers a cluster. This SMT is required for Axual’s topic-name rewriting to function. In imageVolumes mode the chart’s plugins list is empty by default, so it has to name kafka-topic-name-transforms explicitly; every prebuiltImage profile already includes it.

The Connect REST API runs on port 8083 inside the cluster as a ClusterIP service. Kafka Connect’s own Basic Auth cannot be switched on: it needs rest.extension.classes, and Strimzi refuses that setting on purpose, so the check cannot live inside the worker.

Two chart values put it back:

Neither half works alone, so the chart refuses to render a route while restApi.networkPolicy is off.

Register the cluster’s connectUrl in Self-Service as the route address, not the internal Service, and set the authentication method to match restApi.route.authMethod. See REST API access values for the chart side, and Secure the REST API for the procedure.

The Log Provisioner is the Runtime Provisioner, deployed once per Kubernetes cluster and reading worker logs through the same Service that provisions KSML applications. Platform Manager calls it to surface connector logs in the Self-Service UI. Registering the URL against a KC cluster is what gives it that second role, so the Self-Service field keeps the name Log Provisioner.

The Log Provisioner locates the correct pods using three Kubernetes labels that the chart renders automatically:

These label values must match exactly what Self-Service knows about the cluster, in lowercase: the Log Provisioner lowercases the name before it looks for pods, and the chart requires tenant, instance and clusterName to be lowercase label values. A mismatch prevents the Log Provisioner from finding the pods.

Kafka Connect stores its own state in three internal Kafka topics:

It also identifies its worker group to Kafka using a groupId.

If two Kafka Connect clusters share any of these values on the same Kafka broker, they join the same rebalance group and read/write to the same state topics. This corrupts both clusters' state immediately. The chart derives all four from the cluster identity when they are left empty, as <tenant>-<instance>-<clusterName>-connect for the group ID plus -configs, -offsets and -status for the topics, which keeps them unique across a shared broker. The leading keeps them out of the governed <tenant>-<instance>-<environment>- space that user topics live in. Renaming them loses the cluster’s connector configurations, offsets and status, so the chart fails an upgrade that would change them on a running cluster.

ACLs on a Kafka broker grant or deny operations on resources. The worker’s own mutual TLS (mTLS) certificate or Simple Authentication and Security Layer (SASL) username represents its identity on the broker, but that identity cannot grant ACLs to itself. Only a Kafka superuser can modify ACLs.

Broker preparation is therefore a separate operation requiring a superuser identity. The chart’s aclBootstrap Job automates this with a dedicated superuser credential that is separate from the worker identity. The Job runs the Connect base image, not the worker image, because it needs the Kafka command-line tools (kafka-topics.sh and kafka-acls.sh) that the base image ships. In prebuiltImage delivery mode the Job and the worker therefore run different images. See How to Prepare the Kafka Broker for Kafka Connect for the procedure.

Connector credentials (mTLS private keys, SASL passwords) are never stored in Platform Manager’s database. They live in HashiCorp Vault. Two Vault roles serve different purposes:

On Axual Cloud these are often the same Vault server with different secret paths. On strict on-premises setups they can be two separate Vault servers.

The separation is a hard architectural rule, not a preference. On some on-premises deployments the network blocks data-plane-to-control-plane traffic entirely. Connectors running in the data plane must never reach the Governance Vault.

The data plane is not always the same Kubernetes cluster as the control plane. A Kafka Connect cluster can run in a cluster of its own, away from both Platform Manager and Vault, and three connections then cross that boundary: the worker reaches the Connector Vault to resolve its ${vault:…​} placeholders, the worker reaches the Kafka broker, and Platform Manager reaches the cluster’s REST API. Each one has to be reachable, and trusted by whichever side opens it. The Vault leg is the one most often missed, because it is exercised only when a connector is deployed and not when the worker starts. vault.address must also be a hostname carried in the Vault server certificate’s Subject Alternative Name (SAN); an in-cluster service name such as vault.vault.svc is usually absent from the SAN, so it cannot be used from another cluster, nor from the same one while vault.sslVerify is true.

Connector credentials are stored in a Key-Value version 2 (KV v2) secret engine. The path structure is:

With a connector-auth engine mount, a secret/connectors cluster prefix, tenant tenanta, instance ota, cluster kc-cluster, environment env1 and application app1, the first path resolves to connector-auth/secret/connectors/tenanta/ota/kc-cluster/tls/env1/app1.

Only the first two segments are chosen by a person. Platform Manager fills in the rest:

Every connector on one cluster sits under the same prefix, so one Vault policy per cluster covers all of them.

KV v2 internally prefixes all secret paths with data/ for reads and writes, metadata/ for list and metadata-delete operations, delete/ for soft-delete of specific versions, and destroy/ for permanent removal of specific versions. The ${vault:…​} placeholders in connector config omit this prefix because Vault adds it automatically. Vault policies must include it explicitly, for example connector-auth/data/<connector-vault-path>/…​.

The connector-auth mount name is a default, not a fixed name, and every tenant and cluster on one Platform Manager shares whichever name the Operator set. Everything else about a cluster’s Connector Vault is per-cluster: its URL, its AppRole, its optional namespace, and its connectorVaultPath prefix. Two clusters can therefore use entirely different Vault servers while sharing the same mount name. See Platform Manager Vault configuration for the setting itself.

When a Tenant Admin submits a cluster registration in Self-Service, Platform Manager runs three validation checks before accepting the record:

If any check fails, the registration is rejected. All cluster metadata goes to the connect_cluster table.

This validation step brings together all three areas the sections above cover: the cluster’s REST API must be up (runtime), the Vault credentials must be correct (credentials model), and the required SMT must be loaded (plugins).

The pages below install what the sections above describe, in the order they depend on each other:

How to Decommission a Kafka Connect Cluster reverses all four, in the order that leaves nothing behind.