Producing sample data

Preparation

If you intend to try out a Sink Connector for demonstrative purposes, you will need some data to exist on your source-stream.

This short example will show how to produce sample data on any stream, via the "Rest Proxy".

  • You first need to register a producer. You can follow the custom applications documentation if you haven’t created one yet.

  • Know your stream and environment names in advance. You also need your Rest Proxy URL:

    • If you are using Axual Cloud, the rest proxy URL looks like this: https://rest-ams01.cloud.axual.io:<port>. The port is different for every tenant. This information should be available in your internal documentation.

    • If you are using the SaaS trial, your Rest proxy URL is specified in the README.txt included in your care-package.

    • If you are using an Axual-Platform installation on your company’s infrastructure, consult the Axual-Platform operators for the Rest Proxy URL.

    • If you are using the local helm platform-deployment, the URL is https://platform.local:18111

Producing sample data for sink connectors

You can execute a produce request via your Terminal or via Postman.

Producing via the terminal

Acquire the certificate and key file you used when registering your producer. In this example, the two files sit in the working directory and are named auth.key and auth.cert.

Update the first 3 lines in the following script with your deployment information, then run the commands.

REST_PROXY_URL="https://rest-ams01.cloud.axual.io:99999"
AXUAL_ENVIRONMENT="example"
AXUAL_STREAM_NAME="my_test_stream"

curl -k --request POST \
  --url "${REST_PROXY_URL}/stream/${AXUAL_ENVIRONMENT}/${AXUAL_STREAM_NAME}" \
  --header "axual-application-id: irrelevant-for-producers" \
  --header "axual-producer-uuid: ${AXUAL_STREAM_NAME}-rest-producer" \
  --header "Content-Type: application/json" \
  --key ./auth.key \
  --cert ./auth.cert \
  --data '
  {
     "keyMessage":{
        "type":"STRING",
        "message":"{\"name\": \"john\"}"
     },
     "valueMessage":{
        "type":"STRING",
        "message":"{\"name\": \"john\", \"age\": \"28\"}"
     }
  }'

Producing via Postman

  1. Download Postman if you don’t already have it installed.

  2. Acquire the certificate and key file you used when registering your producer.
    You need to configure postman to use these files as a client certificate for Mutual TLS.
    Follow Postman’s official documentation to do so: Manage Postman Certificates

  3. Import this as a postman collection:

    {
	"info": {
		"_postman_id": "b04bd56b-49cf-4c23-8198-2f22fc8d5d72",
		"name": "Axual Platform produce example",
		"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
	},
	"item": [{
			"name": "Produce event",
			"protocolProfileBehavior": {
				"disabledSystemHeaders": {
					"user-agent": true,
					"accept": true,
					"accept-encoding": true,
					"connection": true
				},
				"strictSSL": false
			},
			"request": {
				"method": "POST",
				"header": [
					{
						"key": "axual-application-id",
						"value": "irrelevant-for-producers",
						"type": "default"
					},
					{
						"key": "axual-producer-uuid",
						"value": "{{AXUAL_STREAM_NAME}}-rest-producer",
						"type": "default"
					}
				],
				"body": {
					"mode": "raw",
					"raw": "{\n  \"keyMessage\":{\n    \"type\":\"STRING\",\n    \"message\":\"{\\\"name\\\": \\\"john\\\"}\"\n  },\n  \"valueMessage\":{\n    \"type\":\"STRING\",\n    \"message\":\"{\\\"name\\\": \\\"john\\\", \\\"age\\\": \\\"28\\\"}\"\n  }\n}\n",
					"options": { "raw": { "language": "json" }}
				},
				"url": {
					"raw": "{{REST_PROXY_URL}}/stream/{{AXUAL_ENVIRONMENT}}/{{AXUAL_STREAM_NAME}}",
					"host": [
						"{{REST_PROXY_URL}}"
					],
					"path": [
						"stream",
						"{{AXUAL_ENVIRONMENT}}",
						"{{AXUAL_STREAM_NAME}}"
					]
				}
			},
			"response": []
	}],
	"event": [],
	"variable": [
		{
			"key": "REST_PROXY_URL",
			"value": "https://rest-ams01.cloud.axual.io:99999",
			"type": "default"
		},
		{
			"key": "AXUAL_ENVIRONMENT",
			"value": "example",
			"type": "default"
		},
		{
			"key": "AXUAL_STREAM_NAME",
			"value": "my_test_stream",
			"type": "default"
		}
	]
}

  4. Go to the Variables tab in the collection and update the 3 variables with your deployment information. Don’t forget to save.

  5. Send the Produce event request in order to publish an event.

Reference documentation:

Rest proxy: