MongoDB source Connector, version 2.6.1.Final

MongoDB

If you already have a publicly accessible deployment of MongoDB, you can skip ahead and Configure and install a connector, using the Axual Self-Service UI. Use the configuration instructions below.
If you don’t have one available, the following section instructs you to deploy a publicly available instance of MongoDB.

Deploying a MongoDB instance

  1. Deploy a free shared MongoDB cloud cluster.
    During the "security quickstart" step:

    • You will create a set of credentials used to authenticate against the cluster . Remember these credentials.

    • You will authorize IP addresses to access the cluster. It’s simplest if you authorize 0.0.0.0/0.

  2. Go to "Database". You will see your cluster. There’s 4 buttons after the cluster name, click the …​ (three dots) and then click "Load Sample dataset".

  3. Follow MongoDB instructions and import the dataset.
    Wait for the import operation to complete

  4. Go to Database → Click on the name of your deployment (e.g. "Cluster0"), and click on the primary node.

    Save the cluster name, i.e. ab-cdef1gh-shard-00-01.i2j3klm.mongodb.net. You will use this later as the mongodb.hosts connector-configuration value.

  5. [Optional] If you haven’t done it during the quickstart, you can Go to Network Access and authorize the 0.0.0.0/0 CIDR range.

Configuring a new source Connector

  1. Follow the Creating streams documentation in order to create one stream and deploy it onto an environment.
    The name of the stream will be mongo_nickname.sample_mflix.comments.
    The key/value types will be String/String.

    The stream name follows the following naming convention: <mongo_nickname>.<db_name>.<collection_name>. One such stream is needed for every table we’ll be watching.

  2. Follow the Configure and install a connector documentation to set up a new Connector-Application.
    Let’s call it my_comments_app.
    The plugin name is io.debezium.connector.mongodb.MongoDbConnector.
    If a plugin isn’t available, ask a platform operator to install plugins.

    For advanced configuration, see the {hyperlink-official-source-connector-documentation}.

  3. Provide the following minimal configuration in order to connect to the previously configured MongoDB instance.

    mongodb.hosts
    The hostname of the primary mongodb node, which you saved a few minutes ago

    Example value:
    ac-abcd1ef-shard-00-02.abcdefg.mongodb.net

    mongodb.ssl.enabled

    true

    mongodb.user
    The user you created, NOT your account name

    Example value:
    username

    mongodb.password
    Password for the created user

    Example value:
    password1234

    mongodb.name

    mongo_nickname

    database.include.list

    sample_mflix

    collection.include.list

    sample_mflix.comments

    We need to supply additional (custom) properties, with keys which are not automatically available in the self-service UI. We disable automatic topic creation. The first property should be enough. The fact that the following 2 properties are required is a bug within the connect-plugin itself.

    topic.creation.enable

    false

    topic.creation.default.partitions

    1

    topic.creation.default.replication.factor

    3

  4. Authorize the my_comments_app source Connector-Application to produce to the mongo_nickname.sample_mflix.comments stream.

  5. You can now start the source Connector-Application.

  6. You can now check the mongo_nickname.sample_mflix.comments stream to see the events published by the Connector.

Cleanup

Once you are done, stop the Connector-Application and cleanup the unused axual resources.
In case you deployed your mongo-cluster via MongoDB Cloud, don’t forget to terminate your cluster:

  • Click the cluster

  • Click the …​ three dots on the right side of the screen

  • Click Terminate
    Follow mongodb’s instructions to finalize the termination process.