Configure REST Proxy

This page outlines the configuration values for the REST Proxy deployed via Axual Streaming Helm chart.

About REST Proxy

For detailed information on the REST Proxy, please refer to the REST Proxy documentation.

REST Proxy Configuration

Here you can find some basic configuration and examples that you can use to build your own values.yaml file to deploy REST Proxy. For more details and advanced configuration, please refer to REST Proxy Configuration and REST Proxy Docker Configuration.

REST Proxy repository

First of all you need to add some configuration to specify from where to pull the REST Proxy. You can do this in the following way:

values.yaml

rest-proxy:

  image:
    registry: "docker.axual.io"
    tag: "1.9.3"
  imagePullSecrets:
    - name: docker-credentials

Kafka init container

REST proxy requires an init container (running a Kafka image) to create the ACLs in the Kafka cluster. We need to specify:

The bootstrapServers of the Kafka cluster where we want to apply the ACLs
The principal to whom we want to grant the ACLs. We might need to use the SSL chain to identify the principal or the CN, depending on how the Kafka installation is configured.
The groupPattern which is the group prefix to give access to (typically {tenant}.{instance})
The topicPattern which is the topic prefix to give access to (typically {tenant}-{instance})
The tls Secrets needed to connect to the Kafka cluster

If Kafka is configured to validate ACLs over the full principal chain, please provide the principal chain as this example: [0] CN=Root CA, [1] CN=Intermediate CA, [3] CN=schema-registry. Otherwise, just provide the common name prefixed with CN:.

You can configure it in the following way:

values.yaml

rest-proxy:
  kafkaInitContainer:
    bootstrapServers: ""
    principal: ""
    groupPattern: ""
    topicPattern: ""
    tls:
      keypairSecretName: ""
      keypairSecretKeyName: ""
      keypairSecretCertName: ""
      truststoreCaSecretName: ""
      truststoreCaSecretCertName: ""

Logback Configuration

It is possible to define a ConfigMap containing the logback configuration used by the REST Proxy application. You can configure:

pattern: Defines the exact pattern for log statements
rootLoglevel: Sets the base logging level
loggers: You can configure specific loggers with different levels, overriding the root level for these loggers.

Here is an example:

values.yaml

rest-proxy:

  logging:
    pattern: '%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}'
    rootLoglevel: debug
    loggers:
      io.axual: info
      io.axual.proxy.rest: debug
      org.apache.kafka.clients.admin.AdminClientConfig: info
      org.apache.kafka.clients.producer.ProducerConfig: info
      org.apache.kafka.clients.consumer.ConsumerConfig: info
      org.springframework.boot.web: debug

TLS Configuration

If needed, you can specify secrets containing the PEM certificates for keystore generation:

Server keypair
Client keypair
Truststore

Here is an example of how you can configure it.

values.yaml

rest-proxy:

  tls:
    # -- Creates server keypair from PEM
    createServerKeypairSecret: true
    # -- PEM used to generate the server keypair if `createServerKeypairSecret` is true
    serverCertificatePem: <server-certificate>
    # -- PEM used to generate the server keypair if `createServerKeypairSecret` is true
    serverKeyPem: <server-key>

    # -- Creates client keypair from PEM
    createClientKeypairSecret: true
    # -- PEM used to generate the client keypair if `createClientKeypairSecret` is true
    clientCertificatePem: <client-certificate>
    # -- PEM used to generate the client keypair if `createClientKeypairSecret` is true
    clientKeyPem: <client-key>

    # -- Creates truststore from PEMs
    createTruststoreCaSecret: true
    # -- Set of PEMs used to generate the truststore if `createTruststoreCaSecret` is true
    caCerts:
      ca_one.crt:  <first-cert>
      ca_two.crt: <second-cert>

For more information on the secrets defined above, refer to TLS secrets.

Application Configuration

REST Proxy is a Spring Boot application. Spring Boot application can be configured with application.yml files. What is present under config in the yml file, gets injected in a ConfigMap and mounted as an application.yml file.

values.yaml

rest-proxy:
  config:

Here some configurations of the REST Proxy are presented, the most important ones are going to be described.

Logback and server Configuration

The logback.xml file will be the one that we added before in loggers section. In the server part we have some configuration for the server, like port, the ssl chipers that can be used and the configuration for the Tomcat accesslog, which is disabled by default.

values.yaml

rest-proxy:
  config:
    logging.config: /logging/logback.xml

    server:
      port: 18111
      ssl.ciphers: 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA'
      tomcat:
        accesslog:
          #  defaults to disabled
          enabled: false
          pattern: '{"host": "%h", "timestamp":"%{yyyy-MM-dd HH:mm:ss.SSS}t", "thread": "%I", "request_line": "%r", "response_status_code":"%s", "bytes_sent":  "%b", "request_process_time":"%D","user_agent": "%{user-agent}i"}'
          directory: "/dev"
          prefix: "stdout"
          buffered: false
          suffix: ""
          fileDateFormat: ""

REST Proxy clients Configuration

You can configure the Axual Clients used by the REST Proxy similar to what it is done in the following example.

values.yaml

rest-proxy:

  # -- Configuration passed to the container.
  # Contents get injected to a ConfigMap, which gets mounted as an `application.yml` file.
  config:

    axual:
      tenant: axual
      instance: local
      applicationId: rest-proxy
      applicationVersion: 1.7.0

      sslProtocol: "SSL"
      sslEnableHostnameVerification: false
      acl:
        cacheTtlMs: 30000
        retrySleep: 100
        useCache: false
      producer:
        config:
          # Overrides kafka producer configuration
          metadata-max-age-ms: 180000
          connections-max-idle-ms: 180000
          request-timeout-ms: 120000
          retries: 3
          max-block-ms: 60000
          acks: all
          batch-size: 10
          linger-ms: 1
          max-in-flight-requests-per-connection: 5
          send-buffer-bytes: 10000
          receive-buffer-bytes: 10000
      consumer:
        numberOfThreads: 10
        config:
          # Overrides kafka consumer configuration
          metadata-max-age-ms: 180000
          connections-max-idle-ms: 180000
      avro:
        maxSchemasPerSubject: 100
        basicAuthCredentialsSource: ""

REST Proxy and Discovery API

REST Proxy can be configured to be multi or single cluster. In the case of multi-cluster, the cluster configuration is periodically retrieved from Discovery API and cluster switching is supported.

For a single cluster, the Discovery API endpoint is not needed, but cluster static configuration needs to be provided on startup.

You can set REST Proxy to use the Discovery API in the following way:

values.yaml

rest-proxy:
  config:
    axual:
      config-mode: "discovery"
      endpoint: "<discovery-api-url>"

You can set REST Proxy to not use the Discovery API in the following way:

values.yaml

rest-proxy:
  config:
    axual:
      config-mode: "static"
      static-configuration:
        tenant: ""
        instance: ""
        cluster: ""
        bootstrapServers: ""
        schemaRegistryUrl: ""
        groupIdResolver: "io.axual.common.resolver.GroupPatternResolver"
        groupIdPattern: "{tenant}-{instance}-{environment}-{group}"
        topicResolver: "io.axual.common.resolver.TopicPatternResolver"
        topicPattern: "{tenant}-{instance}-{environment}-{topic}"
        transactionalIdResolver: "io.axual.common.resolver.TransactionalIdPatternResolver"
        transactionalIdPattern: "{tenant}-{instance}-{environment}-{transactional.id}"
        principalBuilderClass: io.axual.security.principal.AdvancedAclPrincipalBuilder