Enabling Connector Logging into Kafka

This guide shows you how to make connector logs visible in Self-Service: enabling support per instance, installing the Axual log appender, and putting Axual Connect into the configuration mode that supports it.

Type

How-to guide

Goal

Get connector logs for an instance visible in the Self-Service interface.

Audience

Platform Operator who can edit the Axual Platform and Axual Connect values and apply them.

When to use

Use this guide when App Owners need to see their connector logs without cluster access.

Axual Connect is deprecated. See Why Kafka Connect replaces Axual Connect for the deprecation timeline. On Kafka Connect, which replaces it, connector logs reach Self-Service through a separate component rather than a log appender in the worker. See How to Enable Kafka Connect Log Reading.

Writing connector logs to a Kafka topic happens only on Axual Connect, and only Axual Connect connectors use the Logs tab that reads those topics, so that tab is deprecated as well. It keeps working for now, but it receives no further development, and it is removed together with Axual Connect. A connector on a Kafka Connect cluster streams its logs straight from the cluster into the log console instead. See Viewing Application Logs and Migrate from Axual Connect to Kafka Connect.

From Axual Platform 2023.2, the Self-Service interface shows connector logging, and a connector can also offload its logging data to an external system. Turning it on takes two changes:

  • Enable connector logging support in the Self-Service interface, per instance.

  • Configure a Kafka log appender for Axual Connect, so Axual Connect writes every log message to a Kafka topic, one topic per connector application per environment.

Connector logging works only on Axual Connect instances that use the static configuration mode. Step 3 sets that mode.
Connector logging works only for new Connector applications.

Contents

The sections below cover each task in this guide:

Prerequisites

Confirm the following before you begin.

Access and permissions required

You need the following access and permissions:

  • Helm access to the namespace Axual Connect runs in, for editing and applying the Axual Platform and Axual Connect values.

  • A Tenant Admin account in Self-Service, for Step 1.

  • Read access to the Kubernetes Secret holding the Axual Connect client certificate, so you can upload it in Step 1.

  • Write access to the web server hosting the plugin and common-resources archives.

Tools and versions required

You need the following tools:

  • helm >= 3.12.

  • kubectl >= 1.28.

  • wget and tar, for rebuilding the common-resources archive.

Resources that must exist before starting

The following must already exist:

Step 1: Enabling connector logging support in Self-Service

To enable connector logging support:

  1. Update the values.yaml for Axual Platform, in the part related to connector logging. Set connectorLoggingEnabled to true and restart the Platform UI service.

    platform-ui:
  config:
    connectorLoggingEnabled: true

  2. Log in to Self-Service as someone with the TENANT_ADMIN role

  3. Go to the respective instance page

  4. Click Edit instance

  5. Confirm Enable connect is on for that instance

  6. Enable Connector Logging for the instance

  7. Upload the certificate Axual Connect uses, which gives Connect the privileges to write to the logging topics. The certificate is in the Secret axual-local-connect-client-certificates, under tls.crt.

  8. Click Update instance to store the configuration

Step 2: Installing the log appender

To view connector logging in Self-Service, install the Axual logging appender, make it available as a library to Connect and configure it in values.yaml. The appender reaches the workers through the commonResourcesFile that axual-connect downloads at startup, so update that archive and the value pointing at it.

Replace every <VALUE> placeholder with your own value before running a command.

  1. Check which reference you have for the commonResourcesFile in the values.yaml that you use for axual-connect-helm

  2. Download the file axual-connect-common-resources-1.0.0.tgz from the artifactsBaseUrl to your local machine and unpack the tarball to a temporary directory, for example:

    mkdir temp-commons
cd temp-commons
wget <URL_TO_COMMONS_FILE>
tar xzf <COMMONS_FILE>
    If you use a different (newer) version of the common resources, download that version instead.

    The directory now holds every common library, ready for a new one.

  3. Add the axual-logging-appender JAR file to the directory and compress the tarball again. The example below gives the URL of the appender.

    cd temp-commons
wget https://stpaxualconnect.blob.core.windows.net/axual-0e0tyou2/axual-logging-appenders-1.0.2.jar
rm axual-connect-common-resources-1.0.0.tgz
tar --disable-copyfile czf axual-connect-common-resources-1.1.0.tgz *

  4. Serve the new tarball (axual-connect-common-resources-1.1.0.tgz) from the same web server as the original one.

  5. Update the values.yaml for axual-connect-helm to use the new tarball. Only commonResourcesFile changes; the other two keys keep the values your installation already uses.

    downloadPlugins:
  artifactsBaseUrl: "<URL_OF_YOUR_FILE_SERVER>"          # keep your existing value
  connectPluginsFile: "<PATH_TO_YOUR_PLUGINS_TARBALL>"   # keep your existing value
  commonResourcesFile: "axual-connect-common-resources-1.1.0.tgz"

Step 3: Configuring Connect

With the logging appender library available to Connect, put Connect in the static configuration mode and enable routedLogging.

  1. In your values file, enable routed logging and switch to the static configuration mode. Which values to use depends on your installation. If you are unsure, contact Axual Support.

    The Discovery API at https://discovery.endpoint/v2 returns the values for the configuration below.

    		 
    axual:
  configMode: "static"
  staticConfig:
    bootstrap.servers: "example.host:9092"
    tenant: "axual"
    instance: "local"
    cluster: "local"
    schema.registry.url: "https://platform.<DOMAIN>:25000"
    group.id.pattern: "{tenant}-{instance}-{environment}-{group}"
    topic.pattern: "{tenant}-{instance}-{environment}-{topic}"
    transactional.id.pattern: "{tenant}-{instance}-{environment}-{transactional.id}"
    enable.value.headers: "false" # leave to default
    group.id.resolver: "io.axual.common.resolver.GroupPatternResolver" # leave to default
    topic.resolver: "io.axual.common.resolver.TopicPatternResolver" # leave to default
    transactional.id.resolver: "io.axual.common.resolver.TransactionalIdPatternResolver" # leave to default

routedLogging:
  enabled: true
  suppressEnvironment: false
  pattern: '%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} %msg'

  2. Issue a helm upgrade command for Axual Connect. Take the chart version from Axual Connect 0.2.1 Helm Readme, whose title states the chart version the page was generated from.

    helm upgrade --install axual-connect oci://registry.axual.io/axual-charts/axual-connect \
  --version <CHART_VERSION> \
  -n kafka \
  -f <YOUR_CUSTOM_VALUES>

  3. Confirm the logging appender initialised. The Connect logging reports it:

    10:19:44,997 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [io.axual.connect.logging.logback.RoutingKafkaAppender]
10:19:44,999 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [KAFKA]
10:19:45,001 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
10:19:45,009 |-INFO in io.axual.connect.logging.logback.RoutingKafkaAppender[KAFKA] - Logger producer properties:

   (configured Kafka producer properties are shown here)

10:19:45,150 |-INFO in io.axual.connect.logging.logback.RoutingKafkaAppender[KAFKA] - Producer configuration complete. Starting appender
10:19:45,150 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.classic.AsyncAppender]
10:19:45,153 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ASYNC_KAFKA]
10:19:45,154 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [KAFKA] to ch.qos.logback.classic.AsyncAppender[ASYNC_KAFKA]
10:19:45,154 |-INFO in ch.qos.logback.classic.AsyncAppender[ASYNC_KAFKA] - Attaching appender named [KAFKA] to AsyncAppender.
10:19:45,155 |-INFO in ch.qos.logback.classic.AsyncAppender[ASYNC_KAFKA] - Setting discardingThreshold to 51
10:19:45,156 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ASYNC_KAFKA] to Logger[ROOT]

  4. Verify the operational changes by creating a connector application, configuring it in an environment and viewing the logging in the Self-Service interface. For more information, see Managing Connector Applications.

Optional: additional Helm configuration

The values below sit under routedLogging in the Axual Connect values, alongside the enabled and pattern keys set in Step 3:

Name Setting Mandatory Default value

enabled

Indicates if connector logging is enabled

no

false

suppressEnvironment

Leaves the environment out of the log topic name when set to true

no

false

pattern

Log message format; connect logging uses Logback Pattern Layout. See the Logback documentation for examples.

yes

none

enableHostnameVerification

Turns off Secure Sockets Layer (SSL) hostname verification when set to false. Some configurations need this, because the verification stops the log appender from producing.

no

true

debugMode

Logs what the log appender is doing. When enabled, the appender logs the Connector context it received, the topic it routed the logging to, and the offset it produced.

The appender logs the connector context and topic per topic until 5 log lines are produced on that topic. In production this can mean a large amount of logging while Connect starts.

no

false

For every other Axual Connect chart value, see Axual Connect 0.2.1 Helm Readme.