Axual Architecture & Components

All interactions with the Governance Layer start with an HTTPS request to the API Gateway.

1 The request arrives at the API Gateway.
2 API Gateway checks authentication and authorization with Keycloak.
3 Keycloak is connected with an Identity Provider (LDAP, EntraID, etc) via OpenID Connect (OIDC).
4 The Keycloak configuration is stored in a database.
5 After interacting with Keycloak, some request bodies are modified (for example, for Topic Browse) to contain everything needed to interact with the Platform UI, Platform Manager (API) and Topic Browse.
6 Platform Manager creates and configures Kafka topics by creating a Kafka AdminClient, and stores schemas in Schema Registries.
7 Platform Manager stores the details in a database.
8 Topic Browse can query topic data of all managed clusters and present it to the end user.

9a, 9b On first startup of the platform, API Gateway starts a wizard process on the Organization Module of Platform Manager to create a Tenant and admin user in Keycloak. This is a one-time process after which the OM can be disabled.

10 A producer or consumer application can reach the Apache Kafka cluster using Kafka protocols on Kafka listeners.
11 Optionally, applications use Schema Registry over HTTPS.
12 Schema Registry stores schemas on a topic.
13 Some applications without Kafka protocol support can use the REST Proxy instead.
14 REST Proxy in turn connects to Kafka.
15, 16 Axual Connect (Kafka Connect) can quickly move data from or into Kafka, and from or into external systems like MQ or a data lake.
17 Security details of Connectors are stored inside Vault.
18 Credentials are written to Vault by Platform Manager.
19 Axual Distributor reads data from Kafka.
20 Distributor writes data into another cluster that is synchronous with the source cluster.

21 The Metrics Exposer gathers metric data from Prometheus and exposes the data for dashboard creation outside the ESP infrastructure.
22 Metrics Exposer calls go through API Gateway before reaching Platform Manager.

23, 24, a client may initiate provisioning of KSML deployments through Platform Manager which in turn will call the KSML Provisioner. KSML Applications are fully capable Kafka Streams applications that interact with Kafka and Schema Registry.

25 Axual MCP allows for interactions with the platform through natural language with the use of your favorite AI agent.