Flink Applications

This guide introduces managing Flink applications: Kafka-to-Kafka streaming transformations, written in Flink SQL, that Axual runs for you on a registered Flink Platform cluster. For what a managed Flink SQL application is next to the other types, see Application types.

Type

Reference

Goal

Find the task or reference you need when configuring and running a Flink application.

Audience

Anyone configuring, running, or sizing a Managed Flink SQL application in Self-Service.

When to use

Start here when Axual runs your Kafka-to-Kafka transformation from Flink SQL.

The pages in this section cover Flink applications:

The procedures shared by every application type apply here too: configuring authentication, requesting topic access, and deleting an application.

Axual Flink Capabilities

Managed Flink SQL currently supports Kafka-to-Kafka streaming only. The table below lists what’s supported and what isn’t yet.

Supported Notes

Supported	Notes
Kafka-to-Kafka streaming with Flink SQL	`kafka` and `upsert-kafka` connectors only.
Auto-generated table DDL	One `CREATE TEMPORARY TABLE` per approved topic subscription, from its latest registered schema.
SASL (Simple Authentication and Security Layer) authentication to Kafka	Certificate authentication (mTLS) and OAuth are not available for this application type.
Deployment lifecycle: create, start, stop, resume, reset, delete	Via the registered Flink Cluster’s REST API.
Resume from the latest checkpoint	Flink native checkpointing; no savepoints.
On-demand status, logs, and metrics	Fetched from Flink when you open the application’s detail page; no background polling.
Deployment size (TaskManager CPU/memory)	See Choosing a Deployment Size.

Kafka-to-Kafka streaming with Flink SQL

kafka and upsert-kafka connectors only.

Auto-generated table DDL

One CREATE TEMPORARY TABLE per approved topic subscription, from its latest registered schema.

SASL (Simple Authentication and Security Layer) authentication to Kafka

Certificate authentication (mTLS) and OAuth are not available for this application type.

Deployment lifecycle: create, start, stop, resume, reset, delete

Via the registered Flink Cluster’s REST API.

Resume from the latest checkpoint

Flink native checkpointing; no savepoints.

On-demand status, logs, and metrics

Fetched from Flink when you open the application’s detail page; no background polling.

Deployment size (TaskManager CPU/memory)

See Choosing a Deployment Size.

Not yet supported Notes

Not yet supported	Notes
Java/JAR-based or Python-based Flink applications	Planned for a future phase.
Non-Kafka connectors (HTTP, databases, filesystem, DataGen, and so on)	Kafka-to-Kafka only.
Savepoint-based stop/resume	Checkpointing only.
Persistent Kafka catalog tables	Only `CREATE TEMPORARY TABLE` is supported.
Multiple Flink Clusters per Instance Cluster	At most one.
GitOps / CI-CD export and import of Flink SQL configurations	Planned for a future phase.
Per-TaskManager logs	Only JobManager logs are available.
Per-vertex job status	Only the aggregate job status is available.
Parallelism configuration	Fixed at 1 per Flink application.

Java/JAR-based or Python-based Flink applications

Planned for a future phase.

Non-Kafka connectors (HTTP, databases, filesystem, DataGen, and so on)

Kafka-to-Kafka only.

Savepoint-based stop/resume

Checkpointing only.

Persistent Kafka catalog tables

Only CREATE TEMPORARY TABLE is supported.

Multiple Flink Clusters per Instance Cluster

At most one.

GitOps / CI-CD export and import of Flink SQL configurations

Planned for a future phase.

Per-TaskManager logs

Only JobManager logs are available.

Per-vertex job status

Only the aggregate job status is available.

Parallelism configuration

Fixed at 1 per Flink application.