Consume Service

Overview

The consume service can be used to consume messages from a stream. The Rest Proxy supports multiple datatypes for key and value. Headers are returned as name and list of not supported.

Lifecycle

For every consume call the REST Proxy unloads the SSL context and runs the ACL checks against the stream on which the consume is made. If access is granted, a consumer will be created internally and cached using the AXUAL-CONSUMER-UUID provided in the HTTP Headers.

The rest of the provided http headers are used to initialize the consumer which then will be reused in subsequent calls (given the same AXUAL-CONSUMER-UUID). That consumer object remains cached for a period of time that is dictated by a REST Proxy configuration parameter (namely axual.consumer.cache-ttl-ms), which defaults to 6 minutes. Consumer objects are evicted from the cache after axual.consumer.cache-ttl-ms of inactivity.

API Specification

HTTP Request Structure

End Point

<baseurl>/stream/{environment}/{stream}

HTTP Method

GET

Content-Type

application/json

Path Variables

Path Variable

Description

environment

The environment short name as specified in the Self Service definition

stream

The stream name as specified in the Self Service definition

HTTP Headers

Header Name

Require / Optional

Description

AXUAL-CONSUMER-UUID

Required

The unique consumer instance id used to identify the consumer object for the session

AXUAL-APPLICATION-ID

Required

The application id of the the consumer application as specified in the Self Service

AXUAL-KEY-TYPE

Required

Contains the type of the key data, this should match with the data used in the stream. Can be NULL, AVRO, AVROJSON or STRING

AXUAL-VALUE-TYPE

Required

Contains the type of the value data, this should match with the data used in the stream. Can be NULL, AVRO, AVROJSON or STRING

AXUAL-COMMIT-STRATEGY

Required

This controls the offset should be committed on next consume call or immediately after the read for the current consume call. Valid values are AFTER_READ and NEXT_CONSUME

AXUAL-APPLICATION-VERSION

Optional

The version of the the consumer application

AXUAL-MAX-MESSAGES-RETURNED

Optional

The maximum number of messages to be returned for the call. Internally this configuration will set the consumer max.poll.records.

AXUAL-AUTO-OFFSET-RESET

Optional

This controls if the stream should be read from the begin (EARLIEST) or the end(LATEST) if no offset has been found. Defaults to EARLIEST

AXUAL-POLLING-TIMEOUT-MS

Optional

The amount of time to wait for the backend poll to complete. During creation of the backend consumer or rebalance this might result in an empty response. This delay during creation/rebalance should normally not take more than 10 seconds. Default value is 500

AXUAL-GET-OFFSETS

Optional

Return consumer assigned partition offsets in response. Valid values are TRUE and FALSE. Default value is FALSE

The UUID needs to be unique (guaranteed by using a UUID generator) and generated prior to the consumer loop. This way the consumer object created internally is unique to the consume session.

HTTP Responses Structure

Successful

HTTP Status Code

200

HTTP Header

Content-Type

application/json

HTTP Body

JSON Object

ConsumeResponse

Error

HTTP Status Code

403

Not authorized to access the stream

HTTP Status Code

404

Stream and environment combination not found

HTTP Status Code

500

Generic Internal Error

HTTP Header

Content-Type

application/json

HTTP Body

JSON Object

AxualRestException