Migrating to axual-kafka charts
Objective
As part of the preparation for the migration from Zookeeper to Kraft, your Kafka cluster should be deployed using axual-kafka
charts, which support Kraft deployment.
Replace axual-streaming with axual-kafka charts
For this step, we are going to replace axual-streaming
charts with axual-kafka
charts, which support Kraft. Your current Chart.yaml
file is using 1.2.2 axual-streaming
charts:
dependencies:
- name: "axual-streaming"
version: "1.2.2"
repository: "oci://registry.axual.io/axual-charts"
condition: "axual-streaming.enabled"
Update the dependencies as follows, using axual-kafka
charts version 0.1.4
:
dependencies:
- name: "axual-kafka"
version: "0.1.4"
repository: "oci://registry.axual.io/axual-charts"
condition: "axual-kafka.enabled"
Migrate cluster configs to axual-kafka charts
Set axual-kafka config
Add the following to axual-kafka
values. This will enable Kafka deployment, add the kafkaNodePool
config, and provide your Kafka cluster name from axual-streaming.kafka.fullnameOverride
value:
axual-kafka:
enabled: true
kafkaNodePools:
kafka:
replicas: <axual-streaming.kafka.kafka.replicas>
resources:
limits:
memory: <axual-streaming.kafka.kafka.resources.limits.memory>
requests:
cpu: <axual-streaming.kafka.kafka.resources.requests.cpu>
memory: <axual-streaming.kafka.kafka.resources.requests.memory>
templ: <axual-kafka.kafka.templ>
roles:
- broker
storage:
type: <axual-streaming.kafka.kafka.storageType>
volumes:
<axual-streaming.kafka.kafka.volumes>
fullnameOverride: <axual-streaming.kafka.fullnameOverride>
If you are using affinity rules for broker scheduling, use the following labels for broker:
|
Set Zookeeper config
-
Copy
axual-streaming.kafka.zookeeper
toaxual-kafka.zookeeper
. -
Add
enabled: true
toaxual-kafka.zookeeper
. -
Add
metrics: true
if you have Kafka metrics enabled (i.e., ifaxual-streaming.kafka.kafka.metrics
istrue
), as it is disabled by default. -
If you have
axual-streaming.kafka.zookeeper.externalLoggingEnabled
, then replace it with the following:values.yamllogging: type: external externalConfig: |- <PLACE_YOUR_LOGGING_CONFIG_HERE>
-
For the pod template, rename the key
axual-kafka.zookeeper.template
toaxual-kafka.zookeeper.templ
.
Set Kafka brokers config
-
Copy
axual-streaming.kafka.kafka
toaxual-kafka.kafka
. -
Remove
axual-kafka.kafka.replicas
,axual-kafka.kafka.resources
, andaxual-kafka.kafka.volumes
, since the number of replicas is coming fromKafkaNodePool
. -
Add
kraft: "disabled"
toaxual-kafka.kafka
. -
If you have
axual-streaming.kafka.kafka.externalLogging
enabled, replace it with the following:values.yamllogging: type: external externalConfig: |- <PLACE_YOUR_LOGGING_CONFIG_HERE>
If no external logging is defined, add the following default logging config to
axual-kafka.kafka
:values.yamllogging: type: inline loggers: kafka.root.logger.level: "INFO" log4j.rootLogger: "INFO" log4j.logger.kafka.controller: "TRACE" log4j.logger.kafka.network.Processor: "FATAL" log4j.logger.kafka.network.RequestChannel$: "WARN" log4j.logger.kafka.request.logger: "WARN" log4j.logger.kafka.server.KafkaApis: "FATAL" log4j.logger.state.change.logger: "TRACE"
-
If you have Kafka
internalListenerTlsEnabled
config, remove it, as the new charts require you to provide internal listener config in the listeners part in the coming steps. -
If you have an internal listener (
internalListenerTlsEnabled
istrue
), provide internal listener config underaxual-kafka.kafka.listeners
:values.yamlinternal: type: internal authentication: type: tls tls: true port: 9093
-
For the external listener with TLS, provide the config under
axual-kafka.kafka.listeners.external
:values.yamlexternal: type: loadbalancer authentication: type: tls tls: true port: 9094 configuration: <copy-of_axual-kafka.kafka.externalListenerConfiguration>
-
If you have any other listeners (
interClusterListener
,scramsha512listener
, andadditionalListeners
), the listeners config will be as follows:values.yamlintcluster: type: loadbalancer authentication: type: tls tls: true port: 9096 configuration: <copy-of_axual-kafka.kafka.interClusterListener.listenerConfiguration> scramsha512: type: loadbalancer authentication: type: scram-sha-512 tls: true port: 9095 configuration: <copy-of_axual-kafka.kafka.scramsha512listener.listenerConfiguration> ## additional listener config <copy-of_axual-kafka.kafka.additionalListeners>
-
Now, you can remove the old definition of the listeners from the values file:
axual-kafka.kafka.externalListenerType
,axual-kafka.kafka.externalListenerConfiguration
,interClusterListener
,scramsha512listener
, andadditionalListeners
. -
For the pod template, rename the key
axual-kafka.kafka.template
toaxual-kafka.kafka.templ
. -
If you have Cruise Control enabled, move it from
axual-streaming.kafka.cruiseControl
toaxual-kafka.kafka.cruiseControl.spec
and addenabled: true
. -
For the security part of cluster and client CAs:
-
Copy
axual-streaming.kafka.kafka.security.clientsCaCert
toaxual-kafka.clientsCaCert.cert.caCrt
. -
Copy
axual-streaming.kafka.kafka.security.clientsCa
toaxual-kafka.clientsCaCert.key.caKey
. -
Copy
axual-streaming.kafka.kafka.security.clientsCaCertGeneration
toaxual-kafka.clientsCaCert.cert.generation
. -
Copy
axual-streaming.kafka.kafka.security.clientsCaGeneration
toaxual-kafka.clientsCaCert.key.generation
.Do the same steps for cluster CA as well. The CA section will look like the following:
values.yamlaxual-kafka: clientsCa: create: true cert: generation: <axual-streaming.kafka.kafka.security.clientsCaCertGeneration> caCrt: | <axual-streaming.kafka.kafka.security.clientsCaCert> extraCerts: <axual-streaming.kafka.kafka.security.extraCaCerts> key: generation: <axual-streaming.kafka.kafka.security.clientsCaGeneration> caKey: | <axual-streaming.kafka.kafka.security.clientsCa> clusterCa: create: true cert: generation: <axual-streaming.kafka.kafka.security.clusterCaCertGeneration> caCrt: | <axual-streaming.kafka.kafka.security.clusterCaCert> key: generation: <axual-streaming.kafka.kafka.security.clusterCaGeneration> caKey: | <axual-streaming.kafka.kafka.security.clusterCa>
-
-
Now, you can delete
axual-kafka.kafka.security
. -
Copy
axual-streaming.kafka.kafka.generateCertificateAuthority
toaxual-kafka.kafka.clientsCa.generateCertificateAuthority
andaxual-kafka.kafka.clusterCa.generateCertificateAuthority
. -
Move
axual-kafka.kafka.podMonitor
toaxual-kafka.podMonitor.kafka
and addenabled: true
underaxual-kafka.podMonitor.kafka
. -
Add the following configuration to
axual-kafka.kafka.kafkaExporter
:values.yamlspec: logging: debug
-
Before deploying your changes, you can expect the following diff:
-
The deployment labels will be updated to use new chart labels.
-
cc-metrics
ConfigMap added. -
kafka-metrics
ConfigMap updated with Kraft config. -
zk-metrics
ConfigMap added. -
If there is any Kafka config diff, update the config until there is no diff.
-
There will be some listener differences; make sure all the listener configurations are present.
-
You can use helm diff to check the diff between what you have changed in your config and what is actually deployed. |
If there is any additional diff, keep updating the config until it is resolved. |
Verification
While the Kafka brokers and Zookeeper are restarting:
-
Watch if the brokers are getting to the ready state.
-
Check if your client applications (producers/consumers) have any issues.
-
Verify that all the listeners are there using the
openssl
command for all listeners.
openssl s_client -connect <listener_host>:<port>
Rollback
If the verification steps are not passed, you can roll back to the axual-streaming
charts by updating the dependencies in the Chart.yaml
file again, as follows:
dependencies:
- name: "axual-streaming"
version: "1.2.2"
repository: "oci://registry.axual.io/axual-charts"
condition: "axual-streaming.enabled"