How to Install Connect Plugins

This guide shows you how to choose a connector plugin, package it the way the download job expects, point Axual Connect at the archive and roll the change out.

Type

How-to guide

Goal

Get a new connector plugin loaded into a running Axual Connect cluster.

Audience

Platform Operator with Helm access to the cluster and write access to the plugin file server.

When to use

Use this guide when adding a connector plugin, or when upgrading one that is already installed.

Axual Connect is deprecated. See Why Kafka Connect replaces Axual Connect for the deprecation timeline. On a new installation use Kafka Connect instead, where plugins are added per cluster rather than shared across every tenant. See How to Deploy a Kafka Connect Cluster.

Contents

The sections below cover each task in this guide:

Prerequisites

Confirm the following before you begin.

Access and permissions required

You need the following access and permissions:

  • Helm access to the namespace Axual Connect runs in.

  • Write access to the file server the plugin archives are hosted on.

Tools and versions required

You need the following tools:

  • helm >= 3.12.

  • tar, for building the archive.

Resources that must exist before starting

The following must already exist:

Choose a plugin

Any Apache Kafka connector plugin works, because Axual Connect is Apache Kafka Connect with Axual multitenancy added. The Connect Plugins Catalog lists the ones Axual has tested.

Check what a plugin needs before installing it. A plugin requiring extra files or environment variables puts those on a shared cluster, where every other tenant on the installation can reach them.

Package and install the plugin

Plugins are not baked into the image. An init job downloads the archives from a file server into a volume at pod startup, and the Axual Connect pods mount that volume. Adding a plugin therefore means changing what the file server serves, then restarting the pods.

The archives are downloaded on every pod restart, so the file server has to stay reachable for the life of the installation, not only at install time.

  1. Build the plugin archive with every plugin directly in its base directory, and add it to the file server alongside the common-JARs archive. There is one archive for plugins and one for common JARs.

    plugins.tgz
  couchbase-kafka-connect-couchbase-4.1.7/
  debezium-connector-mongodb/
  kafka-connect-cassandra-3.0.1-2.5.0-all.jar
  kafka-connect-cosmos-1.14.2-jar-with-dependencies.jar
  README.md

  2. Point Axual Connect at the archives. Take the chart version from Axual Connect 0.2.1 Helm Readme.

    Replace every <VALUE> placeholder with your own value before running a command. 
    downloadPlugins:
  artifactsBaseUrl: "<URL_OF_YOUR_FILE_SERVER>"
  connectPluginsFile: "<PATH_TO_YOUR_PLUGINS_TARBALL>"
  commonResourcesFile: "<PATH_TO_YOUR_COMMON_RESOURCES_TARBALL>"

  3. Roll the change out. The init job downloads the new archives as the pods restart.

    helm upgrade --install axual-connect \
  oci://registry.axual.io/axual-charts/axual-connect \
  --version <CHART_VERSION> \
  --namespace kafka \
  -f <YOUR_VALUES_FILE>

  4. Confirm the plugin loaded by calling the /connector-plugins endpoint on any Connect node’s API. A plugin missing here did not unpack, and the init container’s logs say why.

An installed plugin is not yet selectable by App Owners. See How to Publish Connect Plugins to Self-Service.