Deploying KSML Provisioner

Follow the helm-charts guide to gain access and browse to the ksml-provisioner charts. pull the Charts to get a base values.yaml file to start the installation process.

You could deploy the application using the shipped values.yaml and get a KSML Provisioner based on Helm chart defaults.

The Provisioner requires certain privileges scoped to the namespace in Kubernetes to deploy KSML pod including a ServiceMonitor resource for Prometheus monitoring.

Limitations

  1. The Provisioner can only deploy KSML applications in the same Kubernetes cluster where it is deployed.

  2. The Provisioner can pull charts from OCI-compatible registries only. Nexus registry is not supported.

Required Configuration

Environment Variables Description

NAMESPACE

The Kubernetes namespace where KSML application will be deployed.

REGISTRY_URL

The Helm Chart Registry URL where KSML Helm Charts are pulled from.

CHART_NAME

The name of the KSML Helm Chart.

CHART_VERSION

The version of the KSML Helm Chart.

Optional Configuration

Environment Variables Description

REGISTRY_AUTH_ENABLED

Set to true if Helm Chart registry requires authentication. Default false.

REGISTRY_USERNAME

Username of the Helm Chart Registry defined in REGISTRY_URL.

REGISTRY_PASSWORD

Password of the Helm Chart Registry defined in REGISTRY_URL.

CUSTOM_VALUES_FILE

Path to a custom values file containing extra configurations for KSML deployment. This is useful when configs like resources, securityContext, topologySpreadConstraints etc need to be passed.

CLIENT_CA_FILE

Path to a base64-encoded PEM file containing one or more CA certificates. It is used to validate the Helm Registry’s server certificate.

INSECURE_SKIP_TLS_VERIFY

If true, the validation of Helm Registry’s server certificate. This can lead to man in the middle attacks and should not be used in Production.

DISTRIBUTED_TRACING_ENABLED

If true, enables distributed tracing with OpenTelemetry. To configure the exporter, see section "Enable Distributed Tracing." Default false.

Required Kubernetes Permissions

KSML Provisioner requires a certain set of permissions on the service account to deploy KSML applications. These permissions are described below.

The required service account, role and rolebinding is automatically created by the Provisioner Helm Chart. No special configurations required.

Kubernetes Resource API Group Permissions

configmaps

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

pods

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

pods/log

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

secrets

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

serviceaccounts

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

services

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

statefulsets

apps

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

prometheusrules

monitoring.coreos.com

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

servicemonitors

monitoring.coreos.com

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

ingresses

networking.k8s.io/v1

GET, LIST, WATCH, CREATE, PATCH, UPDATE, DELETE

Provisioning KSML applications in different namespace

By default, KSML applications will be provisioned in the same namespace where the provisioner is deployed. If the applications need to be deployed in a different namespace (eg ksml), pass below values.

env:
  - name: NAMESPACE
    value: "ksml"

rbac:
  namespace: "ksml"

The NAMESPACE environment variable informs the provisioner about where to deploy KSML applications. The rbac.namespace ensures that the provisioner has the necessary permissions to deploy KSML applications in the target namespace.

Bring your own ServiceAccount and RBAC

By default, the helm chart deploys a service account and associated RBAC resources with correct permissions. But if for some reason, you want to use your own service account and RBAC resources, set below configuration:

serviceAccount:
  create: false
  name: my-custom-sa
rbac:
  create: false

A custom service account can be passed via serviceAccount.name. It is your responsibility to ensure this service account has correct permissions to deploy KSML applications (see previous section for exact permissions). The easiest way to do this is to create a Role with relevant permissions and a RoleBinding to associate the role with the service account.

In most cases though, you don’t need this and can keep life simple by having rbac.create: true and let the Helm chart take care of the complexity.

Enable Distributed Tracing

KSML Provisioner supports distributed tracing with OpenTelemetry. To enable, set environment variable DISTRIBUTED_TRACING_ENABLED to true.

To export traces to an OTel backend, configure below environment variables.

Environment Variable Description

OTEL_EXPORTER_OTLP_ENDPOINT

Endpoint of OTel collector. Must start with http/https and include port number.

OTEL_EXPORTER_OTLP_HEADERS

Optional headers.

OTEL_SERVICE_NAME

A unique name to recognize the service.

You can now enable KSML on an Instance through Self Service UI.