Installing the Kafka Sink Connector

Downloading the connector library

The library required to run the connector can be found in the Maven Central
This library also contains the Kafka Source Connector

  1. Search for the artifact on Maven Central

  2. Select the version of the connector you wish to install.

  3. Download the JAR type for your Kafka Connect installation.

    Using the wrong JAR type can result in failing connectors caused by class not found exceptions

    These JAR types are available:

    • jar-with-dependencies
      This version contains the compiled connector code and all the dependencies in the JAR. The Kafka Client included in Connect is used, when Kafka Connect upgrades the producer configuration options will be updated as well.

    • jar
      This version only contains the compiled connector code and none of the potential dependencies. Using this version can cause class not found issues in the future

  4. Continue with Installing the connector

Installing the connector

  1. Determine all machines in the Kafka Connect cluster

  2. Follow these steps for each machine found:

    1. Determine the plugins directory used by Kafka Connect

    2. Create a subdirectory *kafka_synchronisation* in the *plugins* directory

    3. Copy the downloaded connector library to the newly created *kafka_synchronisation* directory

    4. Restart Kafka Connect

    5. Verify that the KafkaSinkConnector is shown by querying the connector-plugins endpoint of the REST API of Kafka Connect on the machine.

      # curl command piped through the JQ tool to format the JSON output
curl -s -X GET http://localhost:8083/connector-plugins | jq -S '.[]'
      The URl used depends on the configuration of your Kafka Connect installation. Additional curl options might be required if basic authentication or TLS is used.