Managing KSML Applications

Creating KSML Applications

Follow the steps outlined in Application Management doc.

Configuring and starting KSML Applications

A KSML Application needs two sets of configuration for every environment in which it’s deployed:

  1. Configuration: Includes the KSML YAML definition and deployment configurations.

  2. Authentication: Same as any other Application, see Authentication configuration.

Axual "Self-Service" UI, section of the "Application" page, showing the security and configuration modal of an unconfigured KSML Application, with the blue circle indicators on both configuration buttons

A blue indicator is shown if any Configuration is still missing.

Configuration

To configure the application, Click on the Gear icon in the application box (hovering on the button will show the text 'Configuration'). This opens the Configuration modal, which contains two tabs: KSML Definition and Deployment:

Axual "Self-Service" UI, "Application" page showing the KSML-definition and Deployment tabs of an unconfigured ksml-application

KSML Definition

In this tab, you can provide the KSML Application Definition in one of two ways:

  • Type directly into the editor inside the modal.

  • Drag and drop an existing YAML file into the modal.

Deployment

In this tab, you can configure the runtime settings for your application, including:

  • Deployment Size

  • Automatic Restart

Deployment Size

This setting allows you to select the CPU and memory profile for your application. Configure it from the Deployment tab in the Configuration modal.

Select your preferred deployment size from the dropdown, or leave it unchanged to use the default setting.

Axual "Self-Service" UI, "Application" page showing the Deployment Size with the default value in the "Deployment" tab

Click Save when done.

If you start the application without selecting a deployment size, it will run using the default profile, shown as (Default) in the dropdown list.

Automatic Restart

This setting determines the approach taken by the provisioner in case the application exits. The two options are On Exit (default) and Never.

  • On Exit results in a Kubernetes StatefulSet resource deployed; this is typical for any streaming application that by definition needs to always be running.

  • Never results in a Kubernetes Job resource deployed; this has a completion state and should be picked if it is unnecessary for this application to be restarted (e.g., a Generator application that produces a given number of messages).

Stateful (Persistent Volumes)

KSML applications that use Kafka Streams state stores can configure persistent storage to preserve application state across restarts.

Configuration Options:

  • Stateful: Enable or disable persistent volume storage for state stores

  • Disk Size: Specify the size of the persistent volume (in MB or GB)

  • State store size cannot be decreased once set - only increases are allowed

  • Changes to state store configuration are only allowed when the application is undeployed

Removing State Store Data

To delete the persistent volume and all stored state:

  1. Ensure your application is in the Undeployed state (click STOP if needed)

  2. Open the Configuration modal and navigate to the Deployment tab

  3. Disable the Stateful option

  4. Click Save

The persistent volume and all state data will be permanently deleted.

Security configuration

Authentication configuration is done in the same way as for any Application type.

Requesting Topic Access

Your KSML Application needs to be authorized to produce/consume a kafka topic before it can be started.
Follow the docs to grant it access.

Application Lifecycle

Applications are helm installed on the Kubernetes cluster where the Instance’s configured Provisioner runs. The Platform Manager automatically polls application status, and the current state is displayed on the Application card.

Available KSML Application Actions

The following actions are available to the Application owner for Applications:

Application action Effect Affected Kubernetes Resources

START

Deploy the KSML application on k8s

Pods, PV, PVC, ConfigMap

STOP

Undeploy the KSML application on k8s

Pods, ConfigMap

Start

When the Application is properly configured, START becomes enabled. Clicking it results in helm deploying the KSML definition you have provided on Kubernetes. The application will transition through different states: first STARTING while Kubernetes provisions resources, then STARTED once the application is running successfully. Within seconds, if all goes well, you should see the application reach the STARTED state.

Axual "Self-Service" UI, "Application" page showing the KSML configuration status box of a configured application, in "running" state

If the status changed to FAILED, you can observe the Application through the logging (see below).

Viewing KSML Logging

As soon as you have started your KSML application, you can consult the KSML Application logs to get a deeper insight into the status of the application beyond the status indicator.

On the KSML Application card, click on the "Logs" button. The following screen shows.

Axual "Self-Service" UI, "Application" page showing the connector logs tab

Log contents can be refreshed or downloaded by hitting the top right Refresh Logs icon or the bottom right Download button respectively.

Viewing KSML Metrics

You can see usage metrics for a KSML application on the "Insights" section of the KSML Application card, and the "Performance" section beneath.

The Insights section shows gauges for CPU and Memory usage based on the maximums defined for the app.

The Performance section displays live updating graphs for CPU and Memory that start empty on first load of the page and build the usage history.

Axual "Self-Service" UI, "KSML Application" page showing the Metrics section

Performance metrics are collected in real-time while the page is open and are not persisted. The user should keep the page open to collect more data which are discarded on closing of the tab.

Stop

Stopping the application can be done by clicking the STOP button on the Application card, which undeploys the application from Kubernetes. The application will be removed from the cluster. If the Stateful option is enabled, the PVC will be left intact.

Deleting A KSML Application

Follow the docs to remove the application.

Deleting the KSML Application results in all KSML deployments being stopped on all environments.

KSML Application States

KSML applications transition through different states based on infrastructure status and user actions. The Platform Manager automatically polls the status from Kubernetes and displays it on the Application card.

State Lifecycle

When you start a KSML application, it follows this lifecycle:

  1. User clicks START

  2. Application shows Starting while Kubernetes provisions resources

  3. Application shows Running once successfully started

  4. Failing (if errors occur) → Application encounters issues

  5. Complete (for Job-type applications) → Application finishes successfully

  6. User clicks STOP → Application is removed

figure

Status Mapping

Self-Service keeps polling the provisioner for the status of the Deployment as long as the Application Overview page is open. The status on the Application card can be one of Running, Starting, Failing or Complete.

Based on the Automatic Restart setting provided the conditions for those statuses are different.

Setting Status Condition

On Exit

Running

StatefulSet is running with ReadyReplicas matching Replicas

Starting

Replicas or ReadyReplicas do not yet match expectations

Failing

StatefulSet is failing and pod is restarting

Undeployed

StatefulSet is not deployed

Never

Running

Job Running with multiple replicas (1 succeeded)

Starting

Replicas or ReadyReplicas do not yet match expectations

Failing

Pod in CrashLoopBackoff state

Complete

All Jobs successfully completed

Undeployed

Job is not deployed