Axual Architecture & Components

This guide explains how the Axual Platform fits together: the layers it is built from, the components in each layer, and the path a request takes as it moves between them.

Type

Explanation

Goal

Build a mental model of the platform accurate enough to reason about a change to it

Audience

Anyone who needs to understand the platform’s structure, whether or not they have access to a cluster

When to use

Before designing an integration, sizing a deployment, or tracing where a request goes

Contents

The sections below cover each area in this guide:

Architecture

The diagram below shows the platform’s components, grouped by layer, and every interaction between them.

Axual Platform component diagram

Each shaded group is one part of the platform. Users and Clients call the platform, Axual Governance and Axual Streaming hold the platform’s own layers, Runtime holds the Axual Runtime components, Kafka holds the brokers and controllers, and Services holds the external systems the platform depends on, such as the Identity Provider, the SQL databases, Vault and the Kubernetes API server.

Interactions

The numbers in the diagram mark each interaction between components. They are grouped below by the part of the platform they belong to.

Self-Service interactions

All interactions with the Governance Layer start with an HTTPS request to the API Gateway.

1 API requests arrive at the Axual API Gateway.
2 Axual API Gateway checks authentication and authorisation with Keycloak.
3 Keycloak connects to an Identity Provider, for example LDAP or Microsoft Entra ID, over OpenID Connect (OIDC).
4 Keycloak stores its configuration in a SQL database.
5 After interacting with Keycloak, some request bodies are modified, for example for Topic Browse, to contain everything needed to interact with the Platform UI, Platform Manager and Topic Browse.
6 Platform Manager creates and configures Kafka topics through a Kafka AdminClient, and stores schemas in the Apicurio Registry.
7 Platform Manager stores topic metadata, users, groups, configs in a SQL database.
8 Topic Browse queries topic data of all managed clusters and presents it to the end user.
9 Platform Manager creates realm and roles in Keycloak

Kafka interactions

The Streaming Layer carries the data itself, reached either over the Kafka protocol or over HTTP.

10 A producer or consumer application reaches the Apache Kafka cluster using Kafka protocols on external Kafka listeners.
11 Applications can use the Apicurio Registry to write data using AVRO, JSON or Protobuf schemas.
12 Apicurio Registry stores schemas on a topic. Schema write permissions set using Apicurio Keycloak and its SQL database
13 Applications without Kafka protocol support use the Rest Proxy instead.
14 Rest Proxy connects to Kafka and Apicurio Registry.
15 Axual Runtime components interact with Apicurio for schemas.
16 Kafka Connect moves data into and out of Kafka and into and out of external systems such as a message queue or a data lake.
17 Platform Manager writes credentials to Vault.
18 Connector security details are obtained from Vault.
19 Axual Runtime components interact with Kafka.
20 Axual Distributor writes data into another cluster that is synchronous with the source cluster, together forming an Axual Instance.

Other interactions

21 Platform Manager registers schemas to Apicurio Registry. Schema registration permissions are configured in Apicurio Keycloak
22 Kafka Connectors are managed through Platform Manager on a target Kafka Connect cluster.
23 Platform Manager calls the Runtime Provisioner to deploy KSML applications and to read Kafka Connect worker logs.
24 The Runtime Provisioner calls the Kubernetes API server to create the KSML application resources and to read Kafka Connect worker logs.
25 Axual MCP allows interactions with the platform through natural language, using an AI agent.
26 The Metrics Exposer gathers metric data from Prometheus and exposes it for dashboard creation in a different datacenter.

Component overview

Each component carries a short identifier, used in Network and Port Reference.

Streaming layer

The Streaming Components hold Axual Kafka, the Kafka distribution the platform deploys, and the components that sit directly in front of it:

Governance layer

The Governance Components hold Self-Service, the management API and their backing services:

Run-time layer

The Runtime Components move and process data:

Axual Kafka

The Axual distribution of Apache Kafka, deployed by its own chart rather than abbreviated in the diagram.

Axual MCP

Axual MCP sits alongside the three layers rather than inside one:

  • MCP - Axual MCP, a natural language interface to the platform through AI agents

Network Interactions

Every network connection the platform makes, the container port it reaches and the protocol it carries are listed in Network and Port Reference, together with example NetworkPolicy manifests.

That reference is the source for firewall rules, Kubernetes NetworkPolicies and service mesh authorisation policies. The interactions numbered above name which components talk to each other; the reference gives the ports and protocols they use.