Schema Registry (Legacy)

Functionality

Overview

The Schema Registry is based off the Confluent Schema Registry. Due to a change in licensing, the component is considered legacy and is succeeded by Apicurio Schema Registry.

About the Schema Registry

The Schema Registry provides client applications with the Avro schemas available in a specific tenant.

The Schema Registry is always connected to a single Kafka Broker in the same cluster to allow for clusters to have different tenant/instance isolation patterns.

Docker Environment Variables

To run the Axual Schema Registry as a Docker container, requires a minimum set of environment variables for proper configuration. The essential variables are outlined below.

For any additional configurations or settings related to Confluent Schema Registry, please consult the Confluent Official website.

Please be aware that the Axual Schema Registry is based on the Confluent Schema Registry client version 4.1.3, so the configuration provided here reflects the available settings of that specific version.

Configuration

Required

Description

Default Value

SCHEMA_REGISTRY_HOST_NAME

Yes

The hostname or IP address of the running Schema Registry.

<no default>

SCHEMA_REGISTRY_LISTENERS

Yes

Comma-separated list of listeners that listen for API requests over either HTTP or HTTPS.

<no default>

SCHEMA_REGISTRY_MASTER_ELIGIBILITY

Yes

Indicates whether the Schema Registry instance is eligible to be the master.

<no default>

SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS

Yes

Comma-separated list of Kafka bootstrap servers for Schema Registry’s Kafka store.

<no default>

SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL

Yes

The security protocol to use when connecting with Kafka.

<no default>

SCHEMA_REGISTRY_KAFKASTORE_TOPIC

Yes

Kafka topic used for storing Schema Registry data.

<no default>

SCHEMA_REGISTRY_KAFKASTORE_GROUP_ID

Yes

Consumer group.id for the Kafka store.

<no default>

SCHEMA_REGISTRY_SCHEMA_REGISTRY_GROUP_ID

Yes

Consumer group.id for the Schema Registry.

<no default>

When running a Schema Registry in a multi-node setup, it’s important to configure certain properties to ensure proper communication and coordination between nodes. The SCHEMA_REGISTRY_SCHEMA_REGISTRY_GROUP_ID environment variable is used to set the Kafka Consumer Group ID for the Schema Registry. In a multi-node setup, each instance of Schema Registry should have a unique Consumer Group ID to ensure that they don’t interfere with each other.

Docker Run Command

The following Docker run command is an example to start the Axual Schema Registry master container with minimum essential environment variables and volume mounts. Replace all placeholders with your actual information when using this command.

Example:

docker run --rm --name doc-axual-sr-master-container \
-e SCHEMA_REGISTRY_KAFKASTORE_SSL_KEYSTORE_LOCATION="/etc/keystores/[KAFKASTORE_KEY_STORE_FILE_NAME]" \
-e SCHEMA_REGISTRY_KAFKASTORE_SSL_TRUSTSTORE_LOCATION="/etc/keystores/[KAFKASTORE_SSL_TRUSTSTORE_LOCATION]" \
-e SCHEMA_REGISTRY_KAFKASTORE_SSL_KEYSTORE_PASSWORD='[KAFKASTORE_SSL_KEYSTORE_PASSWORD]' \
-e SCHEMA_REGISTRY_KAFKASTORE_SSL_KEY_PASSWORD='[KAFKASTORE_SSL_KEY_PASSWORD]' \
-e SCHEMA_REGISTRY_KAFKASTORE_SSL_TRUSTSTORE_PASSWORD='[KAFKASTORE_SSL_TRUSTSTORE_PASSWORD]' \
-e SCHEMA_REGISTRY_SSL_KEYSTORE_TYPE='[SSL_KEYSTORE_TYPE]' \
-e SCHEMA_REGISTRY_SSL_KEYSTORE_LOCATION="/etc/keystores/[SSL_KEYSTORE_LOCATION]" \
-e SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION="/etc/keystores/[SSL_TRUSTSTORE_LOCATION]" \
-e SCHEMA_REGISTRY_SSL_KEY_PASSWORD='[SSL_KEY_PASSWORD]' \
-e SCHEMA_REGISTRY_SSL_KEYSTORE_PASSWORD='[SSL_KEYSTORE_PASSWORD]' \
-e SCHEMA_REGISTRY_SSL_TRUSTSTORE_PASSWORD='[SSL_TRUSTSTORE_PASSWORD]' \
-e SCHEMA_REGISTRY_SSL_TRUSTSTORE_TYPE='[SSL_TRUSTSTORE_TYPE]' \
-e SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS='SSL://localhost:9093' \
-e SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL='SSL' \
-e SCHEMA_REGISTRY_LISTENERS='https://0.0.0.0:8081,http://0.0.0.0:8080' \
-e SCHEMA_REGISTRY_HOST_NAME='10.1.23.86' \
-e SCHEMA_REGISTRY_MASTER_ELIGIBILITY='true' \
-e SCHEMA_REGISTRY_KAFKASTORE_TOPIC='_axual-local-schemas' \
-e SCHEMA_REGISTRY_SCHEMA_REGISTRY_GROUP_ID='axual-local-schema-registry-master-[RANDOM-UUID]' \
-e SCHEMA_REGISTRY_KAFKASTORE_GROUP_ID='axual-local-schema-registry-master' \
-v [LOCAL_DIRECTORY_PATH]:/etc/keystores \
[IMAGE_REGISTRY]/axual/schemaregistry:[IMAGE_TAG]

-v [LOCAL_DIRECTORY_PATH]:/etc/keystores: Mounts the local directory containing SSL keystore and truststore files into the container’s /etc/keystores directory.

axual/schemaregistry:[IMAGE_TAG]: Specifies the Docker image to run.

Enabling authentication

Schema Registry supports two types of authentication

Basic authentication
Mutual TLS

Both Basic and Mutual TLS authentication can be enabled at the same time. Below are the scenarios how clients will be authenticated if both options are enabled.

If mTLS is enabled and set to NEED, client will be authenticated using mTLS only.
If mTLS is enabled and set to WANT and during connection to Schema Registry client provides the certificates, then client will be authenticated using mTLS.
If mTLS is set to WANT and during connection to Schema Registry client don’t provide certificates, then client will be authenticated using Basic authentication.

Enabling Basic authentication

If SASL is enabled, you can enable basic authentication. In this approach each connection to the SSL endpoint must provide a valid username and password. This is disabled by default, but can be activated per cluster.

Using Axual Helm Charts

Open the values.yaml for your instance and add the following configuration and update

platform:
    instance:
      schemaregistry:
        security:
          slave:
            basicAuth:
              enabled: false
              # Override Instance API URL if it runs outside the K8S cluster
              # instanceApiUrlOverride: ""

              # SSL config used to interact with Instance API
              ssl:
                enabled: true
                clientKeystore: "/u3+7QAAAAIAAAABAAAAAQAh..."
                clientKeyPassword: password
                clientKeystorePassword: password
                clientTruststore: "/u3+7QAAAAIAAAAFAAAAAgA..."
                clientTruststorePassword: password

You need to add Schema Registry client certificate CN to Instance-API config security.authorizedDns so that Schema Registry can access Instance-API for authentication.

Example: Assuming Schema Registry CN is Axual Local Schema Registry Client

platform:
  instance:
    instanceapi:
      security:
        enabled: true
        authorizedDns: "Interservice Client, Axual Local Discovery API Client, Axual Local Schema Registry Client"

Enabling authentication with a client certificate (Mutual TLS)

The Schema Registry has support for authentication with Mutual TLS. In this approach each connection to the SSL endpoint must provide a client certificate signed by a certificate authority trusted in the instance. This is disabled by default, but can be activated per cluster.

Using Axual Helm Charts

Open the values.yaml for your instance and add the following configuration and update

platform:
    instance:
      schemaregistry:
        tls:
          slave:
            clientAuth: true
            clientAuthNeed: true

TLS Protocols and Cipher Suites

Want to enable other TLS protocol versions and cipher suites?

From 2021.3, Schema Registry master and slave only accept TLSv1.2 connections by default.
TLSv1.0 and TLSv1.1 are deprecated due to known security vulnerabilities.

If broker is set to use old TLS protocols (like TLSv1 or TLSv1.1) and not TLSv1.2 the Schema Registry connection will fail.
In this case, make sure config SR_MASTER_CLIENT_SSL_PROTOCOLS and SR_SLAVE_CLIENT_SSL_PROTOCOLS (that is used to make connection to brokers) is enabled with broker supported protocol version.

If you still want to enable old TLS protocol versions, you can add it by setting a configuration as mentioned below:

Using Axual Helm Charts

Edit the values.yaml for your instance and add the following configuration

platform:
    instance:
      schemaregistry:
        security:
          slave:
            server:
              # Leave blank to use Jetty’s defaults.
              protocols: TLSv1.2,TLSv1.1,TLSv1
              # A comma-separated list of SSL cipher suites. Leave blank to use Jetty’s defaults.
              cipherSuites:
            client:
              protocols: TLSv1.2,TLSv1.1,TLSv1
              # A comma-separated list of SSL cipher suites enabled for SSL connections to Kafka
              # If not configured, all the available cipher suites are supported.
              cipherSuites:
          master:
            server:
              # Leave blank to use Jetty’s defaults.
              protocols: TLSv1.2,TLSv1.1,TLSv1
              # A comma-separated list of SSL cipher suites. Leave blank to use Jetty’s defaults.
              cipherSuites:
            client:
              protocols: TLSv1.2,TLSv1.1,TLSv1
              # A comma-separated list of SSL cipher suites enabled for SSL connections to Kafka
              # If not configured, all the available cipher suites are supported.
              cipherSuites: