Axual Operator (Strimzi)
About the Axual Operator
The Axual operator is based on Strimzi. Strimzi is a component that deploys and monitors a Kafka cluster. (Zookeeper and Broker). On this page we discuss Strimzi commands which are used often in Kafka administration.
Please also read the Strimzi Operators Guide for some advanced information on administering a Kafka cluster with Strimzi.
Configuring the Number of Kafka Brokers and Zookeeper Nodes
In the helm charts, open the values.yaml in the base directory of platform.
Inside of this file update the value for cluster → kafka → nodes and cluster → zookeeper → nodes . Remember that Zookeeper must always be able to elect a new leader, for that it needs to be able to establish a quorum. A quorum requires an odd number of nodes - 1,2,3,..
cluster:
enabled: true
name: local
kafka:
nodes: 1
replicationFactor: 1
minIsr: 1
zookeeper:
nodes: 1Please never set this to a lower value than configured.
After changing the values please apply the new values. On the command line from the same directory run the command:
helm upgrade --install platform axual-stable/platform -f values.yaml -n kafkaCheck that the nodes have been added by running monitoring the running pods in Kubernetes. It sometimes takes a few minutes to add the new pods
To view an updated list of the zookeeper status:
kubectl get pods -w -n kafka -l app.kubernetes.io/name=zookeeperTo view an updated list of the kafka broker status:
kubectl get pods -w -n kafka -l app.kubernetes.io/name=kafkaBase Kafka Configurations
The base Kafka Broker configuration is found in the file platform/charts/core/charts/kafka/templates. It is probably not necessary to change this file, but please make any changes here if needed:
auto.create.topics.enable: false
unclean.leader.election.enable: false
background.threads: 16
num.replica.fetchers: 4
replica.lag.time.max.ms: 20000
message.max.bytes: 1000012
replica.fetch.max.bytes: 1048576
replica.socket.receive.buffer.bytes: 65536
offsets.retention.minutes: 20160
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
transaction.state.log.num.partitions: 3If you are adding a new broker node to an existing installation then it is also important to add existing partitions to the new broker node. You can accomplish this using (listed in order of least difficult solution to most complex): Cruise Control, Strimzi, or the kafka_topics.sh command.