Managing clusters

Cluster Fields

Name

A unique, human-readable identifier for the Kafka cluster. It helps differentiate between multiple Kafka clusters in environments with multiple deployments. Only letters, numbers and symbols _-. are permitted. Other special characters and spaces are not allowed.

Description

A human-readable description of the Kafka cluster.

Provider Type

The type of Kafka cluster. Currently, only Apache Kafka is supported.

Kafka bootstrap URL

The URL of the Kafka bootstrap server that a client uses to connect to the Kafka cluster initially. The bootstrap URL helps the client to locate the Kafka cluster.

Location

This value indicates where the cluster is deployed, such as "local" for local development environments or a region-specific identifier for different cloud environments.

Skip CA Validation

Specify if the Kafka Admin Client needs to perform CA validation of the Broker server certificate.

Publicly Trusted CA

When a Kafka cluster uses SSL/TLS encryption, certificates issued by a publicly trusted CA ensure secure communication between clients and brokers, preventing the need for manually managing custom certificates.

CA (PEM)

If Publicly Trusted CA is unchecked, configure the private CA certificate to sign the Broker Server certificate.

Authentication method

Specify the mechanism used to authenticate platform manager when connecting to the Kafka brokers. The supported authentication methods are mTLS and SASL (PLAIN, SCRAM_SHA_256, SCRAM_SHA_512). The operator can choose to use mTLS (then they pass cert and private key) or SASL (then they pass username/password).

Shared cluster

Defines the visibility of the cluster. Shared cluster can be used by multiple tenants.

Shared clusters are visible to all tenants defined in the Axual platform. Only SUPER_ADMIN can manage those clusters.

Private clusters are visible to only the tenant that has defined it. TENANT_ADMIN can manage those clusters.

SSL Authentication Mode

Choose between validating clients using either a simple certificate-based approach or a full principal chain. The certificate-based option verifies the client’s identity using just the Distinguished Name (DN). The full principal chain option ensures a more detailed verification by validating the entire chain of certificates, which is recommended for multi-tenant environments. The default value of the SSL authentication Mode is Certificate.

Topic pattern

This pattern is used to determine the actual topic name in the Kafka cluster when a topic is created from Self Service. The default value for a topic pattern is {tenant}-{instance}-{environment}-{topic}

Consumer Group pattern

This pattern is used to determine the actual consumer group name in the Kafka cluster. The default value for a consumer group pattern is {tenant}-{instance}-{environment}-{group}

Transactional ID pattern

This pattern is used to determine the actual transactional IDs in the Kafka cluster. The default value for a transactional ID pattern is {tenant}-{instance}-{environment}-{transactional.id}

The cluster patterns are used to decide how many instances, environments, and tenants can use this cluster.

Advertised Listeners

These are free-form Kafka bootstrap endpoints that can be listed for each type of security protocol. It is shown in the Connectivity Information section of the Application detail page. Users can use it to configure their producers and consumers. This is also helpful if you have different advertised URLs due to networking configurations.

Creating a Cluster

These steps are used to create a Cluster in the Axual Self-Service.

We are writing the steps to add the local cluster started with the Axual Streaming Charts. But these steps can be used as reference for any cluster
  1. Open the clusters menu and press the New Cluster button

  2. Provide the Cluster Details

    1. Input local as the Name

    2. Input Local cluster as the Description

    3. Input local as the Location

    4. Select Apache Kafka as the Provider Type

  3. Press the Continue button

  4. Provide the Kafka Details

    1. Input platform.local:31777 as the Kafka bootstrap URL

    2. Enable the Skip CA Validation

    3. Select TLS as the Authentication Method

    4. Upload a super-user certificate

    5. Upload a super-user private-key

    6. Press the Verify button to validate that the above information is valid

  5. Toggle the Shared cluster to be no

    Shared clusters are visible to all tenants defined in the Axual Platform. Only SUPER_ADMIN can manage these clusters.

    Private clusters are visible to only the tenant that has defined it. TENANT_ADMIN can manage these clusters.

  6. Select the Full Chain Certificate as the SSL Authentication Mode

    The Full Chain Certificate option is recommended for multi-tenant environments.

  7. Press the Add listener button

    1. Input mTLS as the Protocol

    2. Input platform.local:31777 as the URL

  8. Press the Add Cluster button

    Self-Service Add Cluster Advertised listeners section

Editing a Cluster

Tenant Admin can Edit a cluster that they own. To update a cluster:

  1. Visit the Clusters page and click on the cluster you want to edit.

  2. Click on the Edit Cluster button on the bottom right.

    Cluster Edit Page
  3. Fill in or change any information you need and click the Update cluster button.

If you change any of the Kafka bootstrap credential, you will need to Verify them again before you can update.

Deleting a Cluster

Tenant Admin can delete a cluster that they own if there are no active instances defined for it.

  1. Navigate to the edit cluster page and then click the Delete Cluster Button

  2. If there are active instance defined for the cluster, the Delete Cluster Button will be disabled with a hover tooltip:

    Cluster Delete Button Disabled
  3. If there are no active instances defined for the cluster, the following modal will be displayed:

    Cluster Delete Modal

Once you’ve confirmed that you would like to delete the cluster, it will be removed and no longer accessible by any instance.