Kafka Cluster Onboarding Guide

This guide outlines the steps to onboard your Kafka cluster into the Axual Governance using its Wizard.

Prerequisites

  • Make sure that you have a running Axual Governance that you can access.

  • To make the onboarding Wizard visible, you need to configure your values.yaml with skip-onboarding set to false for your Platform-Manager:

platform-manager:
  config:
    axual:
      default:
        skip-onboarding: false

  • For the Kafka cluster, you are onboarding:

    • Connectivity information (endpoint, port)

    • Security information (certificates, SASL credentials)

Kafka and Schema Registry authentication

Depending on the type of authentication which is enabled on Kafka and Schema Registry, there are additional prerequisites. Check the prerequisites for your type of authentication below.

Kafka authentication

Mutual TLS (mTLS) SASL

If you are using mutual TLS (mTLS) to authenticate to Kafka, there are additional prerequisites:

  • The CA certificate which is used to sign the certificate of the broker (PEM) needs to be available

  • The Certificate and Private Key which Platform Manager (Self-Service) API will use to authenticate to Kafka need to be available

  • The Platform Manager certificate DN (Distinguished Name) needs to be added as a superuser in Kafka (see https://kafka.apache.org/documentation/#security_authz)

If you are using SASL to authenticate to Kafka, there are additional prerequisites:

  • The CA certificate which is used to sign the certificate of the broker needs to be available

  • The SASL Username and Password to authenticate to Kafka need to be available

  • The aforementioned username needs to be added as a superuser in Kafka. (See https://kafka.apache.org/documentation/#security_authz)

Schema registry authentication

If you use authentication on the schema registry interface, you need some additional information:

Basic authentication TLS

The username and password to authenticate to Schema Registry

  • The CA certificate (PEM) which is used to sign the certificate of the schema registry needs to be available

  • The Certificate and Private Key which Platform Manager (Self-Service) API will use to authenticate to Schema Registry needs to be available
Axual Governance only supports X.509 certificates in the PEM format and PKCS8 private keys in the PEM format. The extension of certificate and private key can be anything (for example .crt, .pem, .key, .p8 etc).

Initial Setup: Organization Information

  1. Log into the Self-Service interface via the following URL: https://[governance-domain]/login/local. The following screen will be shown

    Keycloak - landing page after first deployment

  2. Click Register to register a Tenant Admin user. This user has administrative privileges on the platform. The following screen will be shown

    Keycloak - adding tenant admin

  3. Register the tenant admin on your platform. Enter details for the following fields

    1. First name

    2. Last name

    3. Email

    4. Username → you will use this to log in next time

    5. Password

    6. Confirm password

  4. Click Register to add the user, the following screen will be shown.

    Wizard Step 1 form

  5. After providing your organization details, click Continue to begin the Kafka cluster onboarding process.

Tenant Shortname cannot be modified/updated after tenant creation

Step 1: Select Your Kafka Provider

You will be presented with options to select a Kafka provider. Onboard my Kafka Cluster – for connecting your own Kafka cluster into Axual governance

  1. Select Apache Kafka under "Onboard my Kafka Cluster".

    Self-Service Onboarding a Kafka Cluster

  2. Click Continue to proceed to the next step.

Step 2: Enter Your Kafka Cluster Details

Fill in the required Kafka connection fields.

Common Fields (Required for all authentication methods)

  1. Kafka bootstrap URL: Provide your Kafka broker in the format host:port

  2. Publicly Trusted CA:

    1. Enable it if your Kafka cluster uses a certificate signed by a trusted public certificate authority.

    2. Disable it if your Kafka cluster uses a custom or private CA.

  3. Authentication method: Select one of the supported authentication options: TLS or SASL

Two authentication flows are supported:

Option 1: TLS Authentication

Use this method when mutual TLS is used for Kafka client authentication.

  1. Select TLS from the Authentication method dropdown

  2. Provide:

    1. Certificate (PEM) – Upload the PEM-encoded client certificates. Must be X.509 formatted and should contain the whole certification chain

    2. Private Key (PEM) – Upload the matching PEM-encoded private key. Must be pkcs8 formatted

  3. Click Verify to validate the TLS credentials

  4. Once verified, click Continue

    Self-Service TLS Authentication

Option 2: SASL Authentication

Use this method if your Kafka cluster uses username/password-based authentication.

  1. Select SASL from the Authentication method dropdown

  2. Select a SASL Mechanism:

    1. PLAIN

    2. SCRAM-SHA-256

    3. SCRAM-SHA-512

  3. Enter the Username

  4. Enter the Password

  5. Click Verify to validate the SASL credentials

  6. Once verified, click Continue

    Self-Service SASL Authentication

Step 3: Import Schema Registry (Optional)

If your setup includes a Schema Registry, click Import Schema Registry.

Provide the following:

  1. REST Endpoint – e.g. http://my-schema-registry-host:20000

  2. Schema Registry Type – Choose between:

    1. Confluent

    2. Apicurio

  3. Publicly Trusted CA – Allow providing a private signing CA

  4. Authentication method – Choose one:

    1. No Authentication

    2. Basic Authentication — Requires username and password

    3. TLS — Requires a client certificate and private key (PEM)

    Self-Service Import Schema Registry

Step 4: Kafka and Schema Discovery

After verifying your Kafka credentials, Axual performs an automatic discovery scan.

The following assets will be detected (if present):

  • Topics

  • Applications (Based on literal ACLs on the imported cluster)

  • Schemas

Self-Service Kafka Discovery

Step 5: Define the Cluster

Define how this cluster will be used:

  1. Usage type – Select one:

    1. Non-production — for dev/test/staging

    2. Production — for critical workloads

  2. Cluster name – Provide a unique, human-readable name for your cluster

Self-Service Cluster Definition

Step 6: Import Existing Topics

Before completing the onboarding process, you will be presented with a summary of actions that Axual will perform.

You will also be able to inspect discovered topics.

Self-Service Discovered Topics

For each discovered topic, you can:

  • Review configuration properties like:

    • Retention policy

    • Cleanup policy

    • Compression type

Self-Service Review Configuration Properties

  • Review authorized applications (ACLs):

    • View roles (e.g., producer, consumer)

    • Link to known applications in your governance system

Self-Service Review ACLs

What does Axual do during onboarding?

Axual performs a read-only import of your Kafka cluster metadata. Specifically, it will:

  • Discover and import:

    • Topic configurations

    • Access Control Lists (ACLs)

    • Schema definitions (latest versions only)

  • Register these assets into your organization’s governance workspace within Axual

Axual does not make any changes to your Kafka cluster, its topics, or your Schema Registry. The process is entirely non-intrusive and read-only.

Known Limitations

Please be aware of the current limitations of the onboarding process:

  • ACLs defined with a prefixed pattern are not imported

  • ACLs for topics that don’t yet exist are ignored

  • ACLs that apply to Kafka Cluster resources are not imported

  • Only the latest version of each schema subject is imported

Step 7: Complete Onboarding

Click Finish to finalize the Kafka cluster import.

Self-Service Onboard the Kafka Cluster

You have now reached the Self-Service dashboard.

Self-Service UI