MQTT source and sink connectors

Sign up for a free trial of Hive MQTT Broker here.

After successful registration, a free cluster should be automatically created in your cluster overview. Note down the broker’s URL and port. we’ll use them later, as part of the connect.mqtt.hosts connector-config.

Click "Manage Cluster" and then click the "Access Management" button in the top-right of the screen. Create a user and note down its credentials: we’ll use them later to connect to the cluster, as the connect.mqtt.username and connect.mqtt.password connector-config.

We’ll use the HiveMQ WebSocket Client to create sample data for our MQTT cluster. Fill the following fields:

Click "Connect".

Expand the "Subscriptions" section. Click "Add New Topic Subscription". In the pop-up window, edit only the "Topic" field: my_mqtt_topic, then click "Subscribe".

Expand the "Publish" section. Use my_mqtt_topic as the topic name (without a trailing '/1') and write some text in the message box, then click "Publish". You should see the message appear in the "Messages" section below.

Keep this page open. We’ll return later to produce more events, after configuring the Connector in the self-service portal.

Follow the 2022.3@axual::self-service/stream-management.html.adoc#creating-streams documentation in order to create one stream and deploy it onto an environment. The name of the stream will be my_mqtt_kafka_topic. The key/value types will be String.

Follow the Configuring Connector-Applications documentation to set up a new connector application.
Let’s call it my_mqtt_app.
The plugin name is "com.datamountaineer.streamreactor.connect.mqtt.source.MqttSourceConnector".
The values you will need to supply as configuration will be listed in this section.
For advanced configuration, see the official connector documentation.
Provide the following minimal configuration in order to connect to the previously configured MQTT instance.

INSERT INTO my_mqtt_kafka_topic SELECT * FROM my_mqtt_topic

Authorize the my_mqtt_app application to produce to the my_mqtt_kafka_topic stream.

You can now start the source and sink Connector-Application.
The connector we just created will not read events produced in the past.

Return to the HiveMQ WebSocket Client tab and produce some more events.
You can now check the stream to see the events published by the connector. The values will be base64 encoded.