CDC — MongoDB Source Connector

Type

source

Class

io.debezium.connector.mongodb.MongoDbConnector

Target System

MongoDB

Maintainer

Debezium / Red Hat

License

Apache License 2.0

Project

github.com/debezium/debezium

Download

Debezium Releases

This page documents version 3.0.8.Final. Newer versions should be compatible unless there are breaking changes, but field names or default values may differ. If you notice discrepancies, please contact Axual Support.

Description

The Debezium MongoDB CDC Source Connector captures change events from MongoDB replica sets or sharded clusters using the MongoDB Change Streams API and publishes them as records to Kafka topics.

It is maintained by the Debezium community as part of the open-source github.com/debezium/debezium.

The connector uses MongoDB Change Streams to tail document-level changes in real time. Change Streams require a replica set or sharded cluster — standalone MongoDB instances are not supported.

Features

  • Real-time Change Data Capture (CDC) from MongoDB collections

  • Captures inserts, updates, and deletes

  • Stream name follows the convention <topic.prefix>.<db_name>.<collection_name>

  • Works with MongoDB Atlas cloud clusters

When to Use

  • You need real-time CDC from MongoDB collections into Kafka.

  • You want to capture all DML events (inserts, updates, deletes) from MongoDB.

When NOT to Use

  • You only need periodic polling — consider a JDBC-style connector instead.

  • You do not have a MongoDB replica set or sharded cluster (standalone MongoDB is not supported).

Installation

The connector is available from the Debezium Releases.

  1. Navigate to the releases page and select the version matching your Kafka Connect installation.

  2. Download the JAR file.

For installation steps, see Installing Connector Plugins.

Configuration

For the complete configuration reference, see the official source connector documentation.

To configure a connector in Axual Self-Service, see Starting Connectors.
For Infrastructure-as-Code deployment, see the Axual Kafka Connect Boilerplates group which contains Terraform and Management API boilerplates for other connectors.

Axual Platform requirements

When running this connector on the Axual Platform, supply the following additional properties to disable automatic topic creation. The connector has a bug that requires the topic.creation.default.* workaround properties even when topic.creation.enable is set to false.

topic.creation.enable

false

topic.creation.default.partitions

1

topic.creation.default.replication.factor

3

Getting Started

Prerequisites

MongoDB instance

  1. Deploy a free shared MongoDB cloud cluster.
    During the "security quickstart" step:

    • Create credentials. Remember these credentials.

    • Authorize IP addresses to access the cluster. Simplest: authorize 0.0.0.0/0.

  2. Go to "Database", find your cluster, click …​ → Load Sample dataset, and wait for import to complete.

  3. Go to Database → click Connect on your cluster → choose Connect your application.
    Copy the connection string (e.g. mongodb+srv://<user>:<password>@ab-cdef1gh.i2j3klm.mongodb.net).
    This is the mongodb.connection.string value.

Axual streams

The stream name follows the convention: <topic.prefix>.<db_name>.<collection_name>.
Create one stream per collection you intend to watch. For this example, create:

  • mongo_nickname.sample_mflix.comments

All streams should use Key/Value type String/String.

See Creating streams if you need to create streams.

Steps

Step 1 — Create a connector application

  1. Follow the Creating streams documentation in order to create one stream and deploy it onto an environment.
    The name of the stream will be mongo_nickname.sample_mflix.comments.
    The key/value types will be String/String.

    The stream name follows the convention: <topic.prefix>.<db_name>.<collection_name>. One stream is needed for every collection being watched.

  2. Follow the Configure and install a connector documentation to set up a new Connector-Application.
    Let’s call it my_comments_app.
    The plugin name is io.debezium.connector.mongodb.MongoDbConnector.
    If a plugin isn’t available, ask a platform operator to install plugins.

Step 2 — Configure the connector

  1. Provide the following minimal configuration:

    mongodb.connection.string

    Example value:
    mongodb+srv://mongo_user:MongoSecret99@ac-abcd1ef.abcdefg.mongodb.net

    topic.prefix

    mongo_nickname

    database.include.list

    sample_mflix

    collection.include.list

    sample_mflix.comments

    Also supply the Axual Platform-required properties described in [_axual_platform_requirements].

  2. Authorize the my_comments_app source Connector-Application to produce to the mongo_nickname.sample_mflix.comments stream.

Step 3 — Start the connector

Start the connector application from Axual Self-Service.

Step 4 — Verify

In Axual Self-Service, use stream-browse on the mongo_nickname.sample_mflix.comments stream to confirm CDC events are arriving.

Cleanup

When you are done:

  1. Stop the connector application in Axual Self-Service.

  2. Remove stream access for the application if no longer needed.

  3. If deployed via MongoDB Cloud, terminate your cluster:

    • Click the cluster → Click the …​ three dots → Click Terminate

    • Follow MongoDB’s instructions to finalize the termination process.

Known limitations

  • Standalone MongoDB instances are not supported — a replica set or sharded cluster is required.

  • topic.creation.enable=false must be set explicitly; the connector has a bug that requires the additional topic.creation.default.* properties as a workaround. See [_axual_platform_requirements].

  • The deprecated mongodb.hosts, mongodb.name, mongodb.user, and mongodb.password properties are replaced by mongodb.connection.string and topic.prefix in Debezium 2.x.

Examples

Minimal configuration

{
  "name": "my-debezium-mongodb-source",
  "config": {
    "connector.class": "io.debezium.connector.mongodb.MongoDbConnector",
    "tasks.max": "1",
    "mongodb.connection.string": "mongodb+srv://mongo_user:MongoSecret99@ac-abcd1ef.abcdefg.mongodb.net",
    "topic.prefix": "mongo_nickname",
    "database.include.list": "sample_mflix",
    "collection.include.list": "sample_mflix.comments",
    "topic.creation.enable": "false",
    "topic.creation.default.partitions": "1",
    "topic.creation.default.replication.factor": "3"
  }
}

License

Debezium source connector is licensed under the Apache License, Version 2.0.