Couchbase Sink Connector

Type

Sink

Class

com.couchbase.connect.kafka.CouchbaseSinkConnector

Target System

Database (Couchbase)

Maintainer

Couchbase

License

Apache License 2.0

Project

github.com/couchbase/kafka-connect-couchbase

Download

GitHub Releases

This page documents version 4.2.8. Newer versions should be compatible unless there are breaking changes, but field names or default values may differ. If you notice discrepancies, please contact Axual Support.

Description

The Couchbase Sink Connector consumes records from Kafka topics and writes them as documents into Couchbase collections.

It is maintained by Couchbase as part of the open-source github.com/couchbase/kafka-connect-couchbase project.

Features

  • Write Kafka records as documents into Couchbase collections

  • Topic-to-collection mapping via couchbase.topic.to.collection

  • TLS support with pre-whitelisted certificate on Axual Connect clusters

  • Configurable document expiry (TTL)

When to Use

  • You need to write Kafka records into Couchbase as part of a data pipeline.

  • You want to materialise a Kafka topic as a Couchbase collection for downstream querying.

When NOT to Use

  • Your Kafka records do not carry a document key — Couchbase requires a unique document ID per record.

Installation

The connector is available from the GitHub Releases.

  1. Download version 4.2.8 of the Kafka connector ZIP.

  2. Extract the JAR files.

For installation steps, see Installing Connector Plugins.

Configuration

For the complete configuration reference, see the official sink connector documentation and the sink configuration options.

To configure a connector in Axual Self-Service, see Starting Connectors. TIP: To speed up your deployment, use the Terraform Boilerplate or the Management API Boilerplate.

Getting Started

This section walks through writing Kafka records into a Couchbase collection.

Prerequisites

Couchbase instance with target collection

You need a running Couchbase cluster reachable from the Kafka Connect cluster, with a bucket and collection to write into.

See the Couchbase Source Connector prerequisites for cluster setup instructions.

Before starting this connector, create a target bucket and collection in Couchbase:

  1. In your Couchbase cluster, go to Clusters → Buckets → Create Bucket. Name it travel-copy.

  2. Within the bucket, create a scope called inventory.

  3. Within the scope, create a collection called airport.

Axual stream with records

The Kafka stream this connector consumes must already exist and contain records. If you followed the Couchbase Source Connector guide, the travel-sample.inventory.airport stream is already populated.

Steps

Step 1 — Create a connector application

Unresolved include directive in modules/ROOT/pages/connect-plugins-catalog/couchbase/sink.adoc - include::../../_includes/create-an-application.adoc[]

Step 2 — Configure the connector

  1. Provide the following minimal configuration:

    connector.class

    com.couchbase.connect.kafka.CouchbaseSinkConnector

    topics

    travel-sample.inventory.airport

    couchbase.bucket

    travel-copy

    couchbase.topic.to.collection

    travel-sample.inventory.airport=inventory.airport

    couchbase.enable.tls

    true

    couchbase.trust.certificate.path

    /home/kafka/certificates/couchbase.pem
    Pre-whitelisted by Axual operators on all Connect clusters.

    couchbase.username

    mytest

    couchbase.password

    Your Couchbase database credential password

    couchbase.seed.nodes

    The Wide Area Network URL from the Couchbase Connect tab

    For advanced options, see the official sink connector documentation and sink configuration options.

Step 3 — Start the connector

Start the connector application from Axual Self-Service. Once running, records from the stream will be written to the inventory.airport collection in the travel-copy bucket.

Step 4 — Verify

In Couchbase, navigate to the travel-copy bucket → inventory scope → airport collection to confirm documents have arrived.

Cleanup

  1. Stop the connector application in Axual Self-Service.

  2. Remove stream access for the application if no longer needed.

  3. Delete your Couchbase cluster if it was created only for testing.

Known limitations

  • TLS is required for Couchbase Cloud connections — ensure couchbase.trust.certificate.path is set correctly.

  • Each Kafka record must carry a document key — records without a key will fail to write.

Examples

Minimal configuration

{
  "name": "my-couchbase-sink",
  "config": {
    "connector.class": "com.couchbase.connect.kafka.CouchbaseSinkConnector",
    "topics": "travel-sample.inventory.airport",
    "couchbase.bucket": "travel-copy",
    "couchbase.topic.to.collection": "travel-sample.inventory.airport=inventory.airport",
    "couchbase.enable.tls": "true",
    "couchbase.trust.certificate.path": "/home/kafka/certificates/couchbase.pem",
    "couchbase.username": "mytest",
    "couchbase.password": "<your-couchbase-password>",
    "couchbase.seed.nodes": "cb.xyz7a3bde91f2.cloud.couchbase.com"
  }
}

License

Couchbase connectors are licensed under the Apache License, Version 2.0.