Configure REST Proxy

This page outlines the configuration values for the REST Proxy deployed via Axual Streaming Helm chart.

About REST Proxy

For detailed information on the REST Proxy, please refer to the REST Proxy documentation.

REST Proxy Configuration

Here you can find some basic configuration and examples that you can use to build your own values.yaml file to deploy REST Proxy. Refer to the REST Proxy 1.12.0 Helm Readme for more details.

REST Proxy repository

First, you need to add some configuration to specify from where to pull the REST Proxy from. You can do this in the following way:

values.yaml

rest-proxy:

  image:
    registry: "registry.axual.io"
    tag: "1.12.0"
  imagePullSecrets:
    - name: docker-credentials

Kafka init container

REST proxy requires an init container (running a Kafka image) to create the ACLs in the Kafka cluster. We need to specify:

bootstrapServers of the Kafka cluster where we want to apply the ACLs
principal to whom we want to grant the ACLs. We might need to use the SSL chain to identify the principal or the CN, depending on how the Kafka installation is configured.
groupPattern which is the group prefix to give access to (typically {tenant}-{instance}- depending on cluster group pattern)
topicPattern which is the topic prefix to give access to (typically {tenant}-{instance}- depending on cluster topic pattern)
tls Secrets needed to connect to the Kafka cluster

If Kafka is configured to validate ACLs over the full principal chain, please provide the principal chain as this example: [0] CN=Root CA, [1] CN=Intermediate CA, [3] CN=schema-registry. Otherwise, provide the common name prefixed with CN:.

You can configure it in the following way:

values.yaml

rest-proxy:
  kafkaInitContainer:
    bootstrapServers: ""
    principal: ""
    groupPattern: ""
    topicPattern: ""
    tls:
      keypairSecretName: ""
      keypairSecretKeyName: ""
      keypairSecretCertName: ""
      truststoreCaSecretName: ""
      truststoreCaSecretCertName: ""

Logback Configuration

It is possible to define a ConfigMap containing the logback configuration used by the REST Proxy application. You can configure:

pattern: Defines the exact pattern for log statements
rootLoglevel: Sets the base logging level
loggers: You can configure specific loggers with different levels, overriding the root level for these loggers.

Here is an example:

values.yaml

rest-proxy:

  logging:
    pattern: '%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}'
    rootLoglevel: debug
    loggers:
      io.axual: info
      io.axual.proxy.rest: debug
      org.apache.kafka.clients.admin.AdminClientConfig: info
      org.apache.kafka.clients.producer.ProducerConfig: info
      org.apache.kafka.clients.consumer.ConsumerConfig: info
      org.springframework.boot.web: debug

TLS Configuration

If needed, you can specify secrets containing the PEM certificates for keystore generation:

Server keypair
Client keypair
Truststore

Here is an example of how you can configure it.

values.yaml

rest-proxy:

  tls:
    # -- Creates server keypair from PEM
    createServerKeypairSecret: true
    # -- PEM used to generate the server keypair if `createServerKeypairSecret` is true
    serverCertificatePem: <server-certificate>
    # -- PEM used to generate the server keypair if `createServerKeypairSecret` is true
    serverKeyPem: <server-key>

    # -- Creates client keypair from PEM
    createClientKeypairSecret: true
    # -- PEM used to generate the client keypair if `createClientKeypairSecret` is true
    clientCertificatePem: <client-certificate>
    # -- PEM used to generate the client keypair if `createClientKeypairSecret` is true
    clientKeyPem: <client-key>

    # -- Creates truststore from PEMs
    createTruststoreCaSecret: true
    # -- Set of PEMs used to generate the truststore if `createTruststoreCaSecret` is true
    caCerts:
      ca_one.crt:  <first-cert>
      ca_two.crt: <second-cert>

For more information on the secrets defined above, refer to TLS secrets.

Application Configuration

REST Proxy is a Spring Boot application. Spring Boot application can be configured with application.yml files. What is present under config in the yml file, gets injected in a ConfigMap and mounted as an application.yml file.

values.yaml

rest-proxy:
  config:

Here some configurations of the REST Proxy are presented, the most important ones are going to be described.

Logback and server Configuration

The logback.xml file will be the one that we added before in loggers section. In the server part we have some configuration for the server, like port, the ssl ciphers that can be used and the configuration for the Tomcat accesslog, which is disabled by default.

values.yaml

rest-proxy:
  config:
    logging.config: /logging/logback.xml

    server:
      port: 18111
      ssl.ciphers: 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA'
      tomcat:
        accesslog:
          #  defaults to disabled
          enabled: false
          pattern: '{"host": "%h", "timestamp":"%{yyyy-MM-dd HH:mm:ss.SSS}t", "thread": "%I", "request_line": "%r", "response_status_code":"%s", "bytes_sent":  "%b", "request_process_time":"%D","user_agent": "%{user-agent}i"}'
          directory: "/dev"
          prefix: "stdout"
          buffered: false
          suffix: ""
          fileDateFormat: ""

REST Proxy client Configuration

You can configure the Kafka Clients instantiated by the REST Proxy similar to what it is done in the following example.

values.yaml

rest-proxy:

  # -- Configuration passed to the container.
  # Contents get injected to a ConfigMap, which gets mounted as an `application.yml` file.
  config:

    axual:
      tenant: axual
      instance: local
      applicationId: rest-proxy
      applicationVersion: 1.12.0

      sslProtocol: "SSL"
      sslEnableHostnameVerification: false
      acl:
        cacheTtlMs: 30000
        retrySleep: 100
        useCache: false
      producer:
        config:
          # Overrides kafka producer configuration
          metadata-max-age-ms: 180000
          connections-max-idle-ms: 180000
          request-timeout-ms: 120000
          retries: 3
          max-block-ms: 60000
          acks: all
          batch-size: 10
          linger-ms: 1
          max-in-flight-requests-per-connection: 5
          send-buffer-bytes: 10000
          receive-buffer-bytes: 10000
      consumer:
        numberOfThreads: 10
        config:
          # Overrides kafka consumer configuration
          metadata-max-age-ms: 180000
          connections-max-idle-ms: 180000
      avro:
        maxSchemasPerSubject: 100
        basicAuthCredentialsSource: ""

REST Proxy static configuration

This segment of the documentation contains the configuration needed to connect to the Kafka cluster and perform group.id and topic resolving. An example follows:

values.yaml

rest-proxy:
  config:
    axual:
      static-configuration:
        tenant: "axual"
        instance: "test"
        cluster: "ams01"
        bootstrapServers: "bootstrap.ams01.cloud.axual.com:9094"
        schemaRegistryUrl: "schema-registry-slave.cloud.axual.com"
        groupIdResolver: "io.axual.common.resolver.GroupPatternResolver"
        groupIdPattern: "{tenant}-{instance}-{environment}-{group}"
        topicResolver: "io.axual.common.resolver.TopicPatternResolver"
        topicPattern: "{tenant}-{instance}-{environment}-{topic}"
        transactionalIdResolver: "io.axual.common.resolver.TransactionalIdPatternResolver"
        transactionalIdPattern: "{tenant}-{instance}-{environment}-{transactional.id}"
        principalBuilderClass: io.axual.security.principal.AdvancedAclPrincipalBuilder