Couchbase Source Connector

Type

Source

Class

com.couchbase.connect.kafka.CouchbaseSourceConnector

Target System

Database (Couchbase)

Maintainer

Couchbase

License

Apache License 2.0

Project

github.com/couchbase/kafka-connect-couchbase

Download

GitHub Releases

This page documents version 4.2.8. Newer versions should be compatible unless there are breaking changes, but field names or default values may differ. If you notice discrepancies, please contact Axual Support.

Description

The Couchbase Source Connector streams document mutations from Couchbase collections and publishes them as records to Kafka topics. It uses the Couchbase Database Change Protocol (DCP) for low-latency, real-time event delivery.

It is maintained by Couchbase as part of the open-source github.com/couchbase/kafka-connect-couchbase project.

Features

  • Stream document mutations (create, update, delete) from Couchbase via DCP

  • Collection-level filtering using couchbase.collections

  • Configurable source handler for raw JSON or structured output

  • TLS support with pre-whitelisted certificate on Axual Connect clusters

When to Use

  • You need real-time streaming of Couchbase document changes into Kafka.

  • You want to replicate Couchbase collections to downstream Kafka consumers.

When NOT to Use

  • You only need periodic batch exports — consider a REST-based integration instead.

  • Your Couchbase version does not support DCP or the Kafka connector.

Installation

The connector is available from the GitHub Releases.

  1. Download version 4.2.8 of the Kafka connector ZIP.

  2. Extract the JAR files.

For installation steps, see Installing Connector Plugins.

Configuration

For the complete configuration reference, see the official source connector documentation and the source configuration options.

To configure a connector in Axual Self-Service, see Starting Connectors. TIP: To speed up your deployment, use the Terraform Boilerplate or the Management API Boilerplate.

Getting Started

This section walks through streaming document mutations from a Couchbase Cloud cluster into an Axual stream.

Prerequisites

Couchbase instance

You need a running Couchbase cluster reachable from the Kafka Connect cluster.

  1. Sign up for a Couchbase Cloud account and deploy a trial cluster.

  2. Import the travel-sample dataset when prompted.

  3. In the cluster, go to Connect → note down the Wide Area Network URL.

  4. Under Manage Allowed IPs, add 0.0.0.0/0 (or restrict to your Connect cluster IP).

  5. Under Database Access → Manage Credentials, create credentials with Read/Write access to all buckets.

Axual stream

The stream where the connector will produce events must already exist in Axual Self-Service. The stream name follows the convention <bucketName>.<scopeName>.<collectionName>. See Creating streams if you need to create it.

Steps

Step 1 — Create a connector application

Unresolved include directive in modules/ROOT/pages/connect-plugins-catalog/couchbase/source.adoc - include::../../_includes/create-an-application.adoc[]

Step 2 — Configure the connector

  1. Provide the following minimal configuration:

    connector.class

    com.couchbase.connect.kafka.CouchbaseSourceConnector

    couchbase.bucket

    travel-sample

    couchbase.enable.tls

    true

    couchbase.trust.certificate.path

    /home/kafka/certificates/couchbase.pem
    Pre-whitelisted by Axual operators on all Connect clusters.

    couchbase.username

    mytest

    couchbase.password

    Your Couchbase database credential password

    couchbase.seed.nodes

    The Wide Area Network URL from the Couchbase Connect tab

    couchbase.source.handler

    com.couchbase.connect.kafka.handler.source.RawJsonSourceHandler

    couchbase.collections

    inventory.airport

    topic.creation.enable

    false
    Add as a custom property — not pre-filled in the configuration form.

    For advanced options, see the official source connector documentation and source configuration options.

Step 3 — Start the connector

Start the connector application from Axual Self-Service. Once running, document mutations in the inventory.airport collection will be published to the travel-sample.inventory.airport stream.

Step 4 — Verify

In Axual Self-Service, use stream-browse on the travel-sample.inventory.airport stream to confirm events are arriving.

Cleanup

  1. Stop the connector application in Axual Self-Service.

  2. Remove stream access for the application if no longer needed.

  3. Delete your Couchbase cluster if it was created only for testing.

Known limitations

  • TLS is required for Couchbase Cloud connections — ensure couchbase.trust.certificate.path points to the correct certificate.

  • One stream per collection is required — the topic name is derived from <bucket>.<scope>.<collection>.

  • topic.creation.enable must be explicitly set to false in Axual environments (Axual manages stream creation).

Examples

Minimal configuration

{
  "name": "my-couchbase-source",
  "config": {
    "connector.class": "com.couchbase.connect.kafka.CouchbaseSourceConnector",
    "couchbase.bucket": "travel-sample",
    "couchbase.enable.tls": "true",
    "couchbase.trust.certificate.path": "/home/kafka/certificates/couchbase.pem",
    "couchbase.username": "mytest",
    "couchbase.password": "<your-couchbase-password>",
    "couchbase.seed.nodes": "cb.xyz7a3bde91f2.cloud.couchbase.com",
    "couchbase.source.handler": "com.couchbase.connect.kafka.handler.source.RawJsonSourceHandler",
    "couchbase.collections": "inventory.airport",
    "topic.creation.enable": "false"
  }
}

License

Couchbase connectors are licensed under the Apache License, Version 2.0.