Axual Kafka

Functionality

Overview

Axual Kafka is a Helm chart designed to deploy and manage a Strimzi-based Axual Kafka cluster with Kraft (Kafka Raft) mode for easy scalability and high performance. This chart simplifies the provisioning of Kafka clusters and related resources, making it highly customizable to suit diverse use cases.

Changelog

More details are in the Axual Kafka Changelog.

Installation

The Axual kafka is a standalone component that is not dependent on another component to start. For installation, refer to the following documentation: Axual Kafka README.

Configuration

Helm configuration

All details of the Helm Chart configurations are in Axual kafka Helm Readme

Logging

Change the logging configuration by following Kafka with External Logging Config

Listeners

Listeners are the network endpoints Kafka brokers use to accept connections from clients. There are internal listeners, used to reach Kafka from inside the same Kubernetes cluster, and external listeners, used to reach Kafka from outside it.

Listeners are defined as named entries under kafka.listeners. The key is a name you choose and the value is passed to the Strimzi Kafka resource without modification, so the listener types, the authentication types and every other field are the ones documented by Strimzi:

Strimzi GenericKafkaListener reference, for the listener fields and the differences between the internal, nodeport, loadbalancer, ingress and route types.
Securing Kafka access, for the authentication types, including SASL SCRAM SHA-512.

A working set of listeners is part of the example values file in Deploying Axual Streaming and Axual Governance on Kubernetes. An inter-cluster listener is needed when the Axual Distributor has to reach another cluster.

When you change the number of broker replicas, remember to add the new brokers to the configuration.brokers list of every external listener.

Certificate Authorities

Two Certificate Authorities are used by the Kafka cluster:

the cluster CA issues the certificates of the Kafka brokers. Clients and other brokers use it to verify a broker’s identity.
the clients CA issues client certificates. Brokers use it to verify the certificates presented by clients.

By default the chart lets Strimzi generate and renew both, through kafka.clientsCa.generateCertificateAuthority and kafka.clusterCa.generateCertificateAuthority. To bring your own CAs instead, set them to false and provide the certificate and the signing key under the top-level clientsCa and clusterCa values, as shown in the example values file in Deploying Axual Streaming and Axual Governance on Kubernetes.

Renewal, validity and the generation counters behave as described in the Strimzi CertificateAuthority reference.

Superusers

Superusers are principals allowed to perform any action regardless of the configured permissions, such as creating and deleting topics or reading and modifying consumer group offsets. The certificates below must be issued by a CA the Kafka brokers trust.

values.yaml

kafka:
  authorization:
    superUsers:
      - "CN=Platform Manager,OU=Axual,O=Axual,C=NL"

When the Axual SSL principal builder is enabled, a principal is identified by its full SSL chain rather than by the subject alone, so superusers are written as a chain:

values.yaml

kafka:
  authorization:
    superUsers:
      - "[0] CN=Root CA, [1] CN=Intermediate CA, [2] CN=Demo Superuser,O=Axual B.V.,L=Utrecht,ST=Utrecht,C=NL"

See Principal Chain building for how the chain is built and how to enable it with axual.sslPrincipalBuilder.

Alerting

The chart can deploy a PrometheusRule for Kafka, enabled with prometheusRule.enabled.kafka. For what the individual alerts mean and how to act on them, see Acting on Alerts.