REST Proxy 1.17.0 Readme
Overview
Provides basic produce, consume and Avro schema resolve capabilities for clients which can’t use the programmatic kafka-client libraries. There are three main flows: * produce a message along with a schema * retrieve a Topic schema IDs * Produce AVRO message along with the schema ID(s) * consume String or Avro messages
The schematics below give a high-level overview of processing; camelcase boxes and messages refer to classes and DTOs.
Produce message and schema
In this case, the full schema for the key and value messages is sent along with the request. Note that the schema will be verified by looking it up in the schema registry; if the schema is not found there the call results in a 404 Not Found.
Actor client
client -> StreamController++: POST ProduceStreamRequest\n(schema + message)
StreamController -> SecurityService ++: isProduceAuthorized
SecurityService --> StreamController --: AuthorizationResult
StreamController -> ProducerService ++: ProduceRequest
ProducerService -> AvroService ++: lookupSchema
AvroService --> ProducerService --: schema ID
ProducerService --> StreamController --: ProduceResponse
StreamController --> client --: ProduceStreamResponseRetrieve schema ID before sending
In this case, first a schema lookup is performed, the retrieved schema
ID is sent along with the message. This results in smaller POST
payloads if more than one message with the same schema is going to be
sent in sequence.
Actor client
Participant AvroController
Participant StreamController
client -> AvroController ++: POST SchemaRequest
AvroController -> SecurityService ++: isRetrieveSchemaAuthorized
SecurityService --> AvroController --: AuthorizationResult
AvroController -> AvroService ++: getSubject
AvroService --> AvroController --: subject (String)
AvroController -> AvroService ++: lookupSchema
AvroService --> AvroController --: id (Optional<Integer>)
AvroController --> client --: SchemaResponse
client -> StreamController++: POST ProduceStreamRequest\n(schema ID + message)
StreamController -> SecurityService ++: isProduceAuthorized
SecurityService --> StreamController --: AuthorizationResult
StreamController -> ProducerService ++: ProduceRequest
ProducerService --> StreamController --: ProduceResponse
StreamController --> client --: ProduceStreamResponseConsume messages
This will return a list of retrieved messages. The polling timeout and maximum number of messages to return can be passed along in the request.
Actor client
client -> StreamController ++: consumeMessages
StreamController -> SecurityService ++: isConsumeAuthorized
SecurityService --> StreamController --: AuthorizationResult
StreamController -> ConsumerService ++: ConsumeRequest
ConsumerService --> StreamController --: List<ConsumedMessage>
StreamController --> client --: List<ConsumedStreamMessage>Loggers
Below is a per-package breakdown of important packages to help operators configure logging:
| Package | Logger | Description |
|---|---|---|
|
Root package |
Application entry point and constants |
|
ACL Service |
Kafka ACL retrieval, caching, and permission validation against topics/consumer groups |
|
Consumer Service |
Kafka consumer lifecycle, message polling, offset commits, and rebalance handling |
|
Exception Handling |
Global REST exception handlers and error response formatting |
|
Producer Service |
Kafka producer lifecycle, message sending with retries, and Avro serialization |
|
Object Registry |
Kafka client (producer/consumer) instance pooling and lifecycle management |
|
Schema Service |
Schema Registry interactions, Avro schema lookup and validation |
|
Security Service |
Authentication (mTLS/OAuth2) and authorization checks for produce/consume/schema operations |
|
Authorizers |
Kafka ACL-based authorizers for produce, consume, and schema retrieval |
|
Serializers |
Custom Kafka serializers/deserializers (JSON, Null) |
|
Topic Controller |
Main REST endpoints
for |
OpenTelemetry Tracing Configuration
The service uses spring-boot-starter-opentelemetry for distributed
tracing.
By default, tracing is enabled with 100% sampling, but OTLP export is disabled.
Enabling Trace Export
To export traces to an OTLP collector (such as Jaeger, Prometheus, or others), configure:
management:
tracing:
export:
enabled: true
opentelemetry:
tracing:
export:
otlp:
endpoint: http://otlp-collector:4317 # Replace with your OTLP collector URL
transport: grpc # Export protocol: grpc or http/protobuf
headers: # Custom HTTP headers you want to pass to the collector, for example auth headers.
key: valueAdjusting Sampling
The trace.sampling.probability property controls the fraction of spans
that are collected. Setting it to 1.0 means all spans will be
exported.
To adjust the sampling rate (default is 100%):
management:
tracing:
sampling:
probability: 0.1 # 10% samplingSupport for Spring Boot 3.5.x properties
The service has support for Spring Boot 3.5.x OpenTelemetry properties.
You can use the following Spring Boot 3.5.x properties to configure OpenTelemetry:
management:
tracing:
enabled: true
otlp:
tracing:
endpoint: http://otlp-collector:4317 # Replace with your OTLP collector URL
transport: grpc # Export protocol: grpc or http/protobuf
headers: # Custom HTTP headers you want to pass to the collector, for example auth headers.
key: valueThe service will map each Spring Boot 3.5.x property to the corresponding Spring Boot 4.x OpenTelemetry property.
| Spring Boot 3.5.x property | Spring Boot 4.x property | Default value |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|