Redis sink connector, version 4.0.0

Redis

If you already have a publicly accessible deployment of Redis, you can skip ahead and create a Connect Application, using the Axual Self-Service UI. Use the configuration instructions below.
If you don’t have one available, the following section instructs you to deploy a publicly available instance of Redis.

Deploying a Redis instance

  1. Sign up for a free trial account with Redis Cloud. You can simply sign up with Google.
    Select your cloud provider. At the time of writing of this documentation, only AWS had European regions available.
    Click "Let’s start free". A Redis instance will be created for you.

  2. Go to the Subscriptions page. There should be a subscription in your name.

  3. Click the database with your name, which ends in -free-db.
    A detail page will open.

    • Note down the Database name. We’ll use it later when configuring the Connector-Application, for the connect.redis.kcql property.

    • Note down the Public endpoint. We’ll use it later when configuring the Connector-Application, for the connect.redis.host and the connect.redis.port properties.

    • Scroll down until the Default user password field. Click the Copy button next to the password.
      Note down the password. We’ll use it later when configuring the Connector-Application, for the connect.redis.password property.

Configuring a new sink Connector

  1. Follow the creating topics documentation in order to create one topic and deploy it onto an environment.
    The name of the topic will be my_redis_stream.
    The key/value types will be JSON/JSON.

  2. Example: Producing data with Rest Proxy as JSON/JSON events to this topic.
    It’s not important what message-key you use: we only care about the value.

  3. Follow the documentation to set up a new Connector-Application.
    Let’s call it my_redis_sink.
    The plugin name is com.datamountaineer.streamreactor.connect.redis.sink.RedisSinkConnector.
    If a plugin or version isn’t available, ask a platform operator to make it available.

  4. Provide the following minimal configuration in order to connect to the previously configured Redis instance.
    For advanced configuration, see the official sink connector documentation.

    topics

    my_redis_stream

    connect.redis.host
    _You noted down the Public endpoint at the beginning. This is the first part of the URL (before the : character)

    Example value:
    redis-12345.a123.eu-central-1-1.ec2.cloud.redislabs.com

    connect.redis.port
    _You noted down the Public endpoint at the beginning. This is the last part of the URL (after the : character)

    Example value:
    15287

    connect.redis.password
    The password for the default user, which you noted down at the beginning.

    Example value:
    1234567890ABCDefghijklmnopqrstuvwxyz

    connect.redis.kcql
    This is the command that translates kafka events to Redis entries.
    You need to include the database name in this query. You noted it down at the beginning.

    Example value:
    INSERT INTO YOUR-DATABASE-NAME SELECT * from my_redis_stream PK name
    If your JSON message has different field names, replace name with the json field name that you want to use as the Redis key-name.

  5. Authorize the my_redis_sink sink Connector-Application to consume the my_redis_stream topic.

  6. You can now start the sink Connector-Application.

  7. You can check if the events you produced to kafka made their way into Redis by checking the database dashboard in your browser.

    • Open the Databases page and click your database.

    • Click the Metrics tab.

    • Look for the graph titled Total keys. You should see that number increase from zero.

Cleanup

Once you are done, stop the Connector-Application and cleanup the unused axual resources.

Cleanup of Redis resources:

  1. Go to the Subscriptions page and scroll down to the Danger zone. Expand it.

  2. Delete database: Click the Delete button.

  3. When prompted, select Delete both in order to delete both the database and your subscription.

License

Redis sink-Connector is licensed under the Apache License, Version 2.0.

Source code

The source code for the Connect-Plugin can be found at github.com/lensesio/stream-reactor.