MongoDB Source Connector

Type

source

Class

com.mongodb.kafka.connect.MongoSourceConnector

Target System

MongoDB

Maintainer

MongoDB

License

Apache License 2.0

Project

github.com/mongodb/mongo-kafka

Download

MongoDB Kafka Connector installation

This page documents version 2.1.0. Newer versions should be compatible unless there are breaking changes, but field names or default values may differ. If you notice discrepancies, please contact Axual Support.

Description

The MongoDB Source Connector streams data from MongoDB collections into Kafka topics using MongoDB Change Streams.

It is maintained by MongoDB as part of the open-source github.com/mongodb/mongo-kafka.

The connector watches MongoDB change streams, a native MongoDB feature that provides an ordered, resumable stream of change events. Unlike the Debezium MongoDB CDC Connector, this connector is maintained directly by MongoDB and uses the official MongoDB Kafka driver.

Features

  • Stream change events from MongoDB collections using MongoDB Change Streams

  • Captures inserts, updates, replaces, and deletes

  • Configurable pipeline for filtering change stream events

  • Supports full document lookup on updates

  • Resumable — stores resume tokens to survive connector restarts

When to Use

  • You need to stream MongoDB change events into Kafka.

  • You want the officially maintained MongoDB connector rather than the Debezium variant.

  • You need full document content on update events (fullDocument: updateLookup).

When NOT to Use

  • You need CDC from a MongoDB replica set without change streams enabled — use the Debezium MongoDB CDC Connector instead.

  • You only need a one-time bulk export of MongoDB data.

Installation

The connector is available from the MongoDB Kafka Connector installation.

  1. Follow the MongoDB documentation to download and install the connector JAR.

For installation steps, see Installing Connector Plugins.

Configuration

For the complete configuration reference, see the official source connector documentation.

To configure a connector in Axual Self-Service, see Starting Connectors.

Getting Started

Prerequisites

MongoDB instance

You need a running MongoDB replica set or Atlas cluster reachable from the Kafka Connect cluster. MongoDB Change Streams require a replica set — standalone MongoDB instances are not supported.

Axual stream

The stream where the connector will produce events must already exist in Axual Self-Service. See Creating streams if you need to create it.

Steps

Step 1 — Create a connector application

  1. Follow the Configure and install a connector documentation to set up a new Connector-Application.
    Let’s call it my_mongodb_source_app.
    The plugin name is com.mongodb.kafka.connect.MongoSourceConnector.
    If a plugin isn’t available, ask a platform operator to install plugins.

Step 2 — Configure the connector

  1. Provide the following minimal configuration:

    connection.uri

    mongodb://username:password@mongodb.example.com:27017/?replicaSet=rs0

    database

    Your MongoDB database name

    collection

    Your MongoDB collection name

    topic.prefix

    mongodb_orders

    key.converter

    org.apache.kafka.connect.storage.StringConverter

    value.converter

    org.apache.kafka.connect.json.JsonConverter

    value.converter.schemas.enable

    false

    For advanced options, see the official source connector documentation.

  2. Authorize the my_mongodb_source_app source Connector-Application to produce to the mongodb_orders stream.

Step 3 — Start the connector

Start the connector application from Axual Self-Service.

Step 4 — Verify

In Axual Self-Service, use stream-browse on mongodb_orders to confirm events are arriving when documents are inserted or updated in MongoDB.

Cleanup

When you are done:

  1. Stop the connector application in Axual Self-Service.

  2. Remove stream access for the application if no longer needed.

Known limitations

  • MongoDB Change Streams require a replica set — standalone MongoDB deployments are not supported.

  • Resume tokens are stored in Kafka Connect’s offset storage. If offsets are lost, the connector restarts from the beginning or a configurable start timestamp.

Examples

Minimal configuration

{
  "name": "my-mongodb-source",
  "config": {
    "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector",
    "connection.uri": "mongodb://username:password@mongodb.example.com:27017/?replicaSet=rs0",
    "database": "my_database",
    "collection": "orders",
    "topic.prefix": "mongodb_orders",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable": "false"
  }
}

License

MongoDB source connector is licensed under the Apache License, Version 2.0.