IBM MQ source Connector, version 1.3.1

IBM MQ

If you already have a publicly accessible deployment of IBM MQ, you can skip ahead and Configure and install a connector, using the Axual Self-Service UI. Use the configuration instructions below.
If you don’t have one available, the following section instructs you to deploy a publicly available instance of IBM MQ.

Deploying an IBM MQ instance

We’ll deploy an IBM MQ instance using IBM Cloud services.
You will get free credit upon Registration.
If you already have an IBM Cloud account, then simply Login.

  1. Create an MQ instance

    1. Location: Frankfurt (eu-de)
      The location isn’t very relevant, but usually you would select a region geographically closest to the Connect-Cluster.

    2. Pricing plan: Lite
      At the time of writing this document, the Lite plan is free, making it ideal for our test.

    3. Configure your resource:

      • Service name: my_axual_mq

      • Everything else (Resource group, Tags, Access management tags) can keep their default values.

    4. In the bottom right corner of the screen, check the License Agreement checkbox and click Create.
      The MQ’s page should open. From this page we can create the queue manager.

  2. Let’s create a queue manager.
    Make sure that the "Queue managers" tab is selected.
    On the right side of the screen, click the Create + button.

    IBM MQ resource page, showing a new MQ-Service with no queue managers configured

    1. A new page opens. Fill in the following queue manager details:

      • Queue manager name: my_axual_queue_manager

      • Display name: my_axual_queue_manager

      • Location: IBM Cloud Germany (should be the default/only one available, if you chose Frankfurt as the region)

      • Size: Lite (keeping the cost free)

    2. Click the Create button at the bottom right of the screen.

    3. A pop-up prompts you to review your changes. Click Continue.
      Your queue manager is being created. Wait for the operation to complete.
      In the next few steps we’ll download some files which will assist in connecting to the MQ.

  3. After your queue manager is created, click it to open its page.
    Click the Connect information button on the right side of the screen.

    IBM MQ-Service QMGR page, showing a new queue manager
    IBM MQ QMGR connection info pop-up

  4. Select JSON CCDT. The file will be downloaded to your machine.
    Click Close to close the pop-up.

  5. Click the Key store tab and wait for the default qmgrcert to load.

    IBM MQ QMGR Key-store page, showing a default certificate

  6. Click the 3 dots at the top-right of the default certificate to open a new menu.
    Click the Download public certificate button and wait for the operation to complete.

  7. The next step is to create application credentials. For that, return to the MQ overview:

    1. Go to the Resources list page.

    2. Expand the "Integration" section.

    3. Click the my_axual_mq MQ service.

    4. Click the Application credentials tab.

  8. Create application credentials by clicking the Add + button.

    1. Display name: my_mq_connector

    2. Description: "Credentials used by an MQ connector application which is deployed within the Axual platform."

    3. Application username: axualconnect

  9. Click Add and generate API key.

  10. Click the Download button on the bottom-right of the pop-up window.

Configuring a new sink Connector

Contact Axual for support with setting up this connector.

  1. Follow the Creating streams documentation in order to create one stream and deploy it onto an environment.
    The name of the stream will be my_mq_stream.
    The key/value types will be JSON/JSON.

  2. Follow the Configure and install a connector documentation to set up a new Connector-Application.
    Let’s call it my_mq_app.
    The plugin name is com.ibm.eventstreams.connect.mqsource.MQSourceConnector.
    If a plugin isn’t available, ask a platform operator to install plugins.

    For advanced configuration, see the official source connector documentation.

  3. Provide the following minimal configuration in order to connect to the previously configured IBM MQ instance.

    mq.ccdt.url
    Path to a file on all connect-nodes which contains connectivity info. You downloaded this file

    Example value:
    file://etc/ibm_mq_ccdt/my_mq_app/ibm-mq-ccdt.json

    mq.password
    This value can be found in the applicationApiKey.json file you just downloaded.

    Example value:
    ABcdefgHIJkLMNOpqRStuvwXYZ1234567890over_flow

    mq.ssl.keystore.location
    Path to a keystore on disk. It’s built from files you downloaded, and stored on the disks of all connect nodes by axual-platform operators.

    Example value:
    /etc/tls/my_mq_app/ibm_mq/keystore.jks

    mq.ssl.keystore.password

    Example value:
    password

    mq.ssl.truststore.location
    Path to a truststore on disk. It’s built from files you downloaded, and stored on the disks of all connect nodes by axual-platform operators.

    /etc/tls/my_mq_app/ibm_mq/truststore.jks

    mq.ssl.truststore.password

    Example value:
    password

    mq.queue.manager

    my_axual_queue_manager

    mq.user.name

    axualconnect

    mq.queue

    my_mq_stream

    topic

    my_mq_stream

    mq.record.builder

    com.ibm.eventstreams.connect.mqsource.builders.JsonRecordBuilder

    The following optional configs are also recommended:

    errors.log.enable

    true

    errors.log.include.messages

    true

    errors.tolerance

    all

  4. Authorize the my_mq_app source Connector-Application to produce to the my_mq_stream stream.

  5. You can now start the source Connector-Application.

  6. You can now check the my_mq_stream stream to see the events published by the Connector.

Cleanup

Once you are done, stop the Connector-Application and cleanup the unused axual resources.
In case you deployed your MQ via IBM Cloud, don’t forget to delete your queue manager:

  1. Go to the Resources list page.

  2. Expand the "Integration" section.

  3. Click the my_axual_mq MQ service.

  4. On the right side of the screen, click the delete button

  5. Confirm deletion

License

IBM MQ source-Connector is licensed under the Apache License, Version 2.0.

Source code

The source code for the Connect-Plugin can be found at github.com/ibm-messaging/kafka-connect-mq-sink