Security

TLS

Mutual TLS

Mutual TLS (mTLS) is the default method used to authenticate clients connections to Apache Kafka, Schema Registry and other platform components. It ensures that the parties at each side of the network connection are who they claim to be by verifying that they use the correct private key.

Axual components work with TLSv1.2

Server Keystore

This is used by the component to host a TLS endpoint (TCP or HTTP). If it is an HTTPS endpoint, then the key pair inside this keystore must have CN value matching the FQDN which will be used to access this endpoint.

Server Truststore

This is used by the component to validate the client certificate presented by the clients connecting to this component. It should not have key pairs but only public certificates (CA or otherwise).

Client Keystore

Key pair inside this keystore is presented by the component when trying to access other components. Only required when other components require a client certificate to be presented.

Client Truststore

If other components endpoint are also SSL, then this truststore is used to validate the trust chain of remote endpoint. Only required when other components endpoint are TLS encrypted.

TLS certificate requirements

For creating PKI certificates (CA), we prescribe using the following key length:

Server certificates: 4096 bits
others: at least 2048 bits

The following set of ciphers are considered secure and should be used:

TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384

Private keys

Axual Governance only supports X.509 certificates in the PEM format and PKCS8 private keys in the PEM format. The extension of certificate and private key can be anything(for example .crt, .pem, .key, .p8 etc).
The starting line of a PKCS8 private key is -----BEGIN PRIVATE KEY-----. So other private keys that start with lines like -----BEGIN RSA PRIVATE KEY----- or -----BEGIN ENCRYPTED PRIVATE KEY----- would not work.

TLS Secrets on Kubernetes

Axual services often require TLS to provide secure network connections. This methodology relies on certificate files being present in the client and server services on startup time to safely guarantee the identity of the corresponding parties.

When on kubernetes, the existence of secrets in a specific format is sufficient to install our services, since Axual Keystore Provider is part of the deployment of our services. Keystore Provider is an initContainer tasked with the creation of the files given that the TLS secrets are in a specific expected format, this format will be clarified in this segment.

Secrets of type `kubernetes.io/tls`

Kubernetes uses a Secret of type kubernetes.io/tls to store TLS data in a standard Secret format.
This type of Secret can be generated by cert-manager.

apiVersion: v1
kind: Secret
metadata:
  name: example-secret
type: kubernetes.io/tls
data:
  ca.crt:  ... (1)
  tls.crt: ... (2)
  tls.key: ... (3)

1	Base64 encoded root CA certificate string
2	Base64 encoded certificate string
3	Base64 encoded private key string

Certificate files

For Axual Platform components, the ca.crt section of above TLS Secret is not often used, so a Client or Server certificate secrets can consist of two data entries. An example of a valid certificate Secret follows:

apiVersion: v1
kind: Secret
metadata:
  name: example-secret
type: Opaque
data:
  tls.crt: ... (1)
  tls.key: ... (2)

1	Base64 encoded certificate string
2	Base64 encoded private key string

Truststore (CA) files

Truststore (CA) certificate secrets consist of an arbitrary number of key, value pairs, each of which is appended to the resulting truststore that gets mounted onto the service using it.

All CA certificate files must have a .crt extension.

An example of a valid CA secret follows:

apiVersion: v1
kind: Secret
metadata:
  name: example-ca-certificates
type: Opaque
data:
  axual_root_ca.crt: ...
  axual_intermediate_1_ca.crt: ...
  axual_intermediate_2_ca.crt: ...

In the above example, all the abbreviated string values are Base64 encoded strings.

Kubernetes

Kubernetes itself does not provide tools to inspect Secret details, but you can use the following to get certificate details, base64 decode it and then inspect it via openssl.
This example is to inspect the Root CA of Secret TLS_SECRET_NAME:

kubectl get secret TLS_SECRET_NAME -o 'jsonpath={.data.ca\.crt}' | base64 -d | openssl x509 -subject -issuer -startdate -enddate -noout

Using Certificates in Axual Platform

In the Helm installation the certificates are defined in the values.yaml configuration for each service in the tls settings. Given the Helm setup is only necessary to enter the pem for the truststore, client and server keystores.

Truststore

It is necessary to have a ca truststore. This store contains a list of the trusted certificate roots. The server certificates are used to secure the services of each module and the client truststore is used for connections to other services. For the internal components, it makes sense to define a single truststore Secret and use the same truststore for all components.
Refer to this to create a Truststore

Certificates

Since every component requires certificates for mTLS and because certificates need to be re-issued often, it is a high operational burden to manually issue certificates.
Axual highly suggests using cert-manager to automatically issue certificates and reloader to restart Pods after certificate (re)issue.

Configuring certificates

To configure certificates on Axual components, enable tls in the values.yaml and refer to existing Secrets that are the result of Certificates issued by cert-manager.

tls:

set enabled to true
set clientKeypairSecretName to the name of the component Secret of type: kubernetes.io/tls
set serverKeypairSecretName to the name of the component Secret of type: kubernetes.io/tls (it can be the same as the clientKeypairSecretName)
set truststoreCaSecretName to the internal truststore.

Helm Schema for reference:

"tls": {
  "type": "object",
  "required": [
    "enabled",
    "clientKeypairSecretName",
    "serverKeypairSecretName",
    "truststoreCaSecretName"
  ],
  "properties": {
    "enabled": {
      "type": "boolean",
      "description": "Specifies whether a TLS should be configured",
      "default": "false"
    },
    "clientKeypairSecretName": {
      "type": "string",
      "description": "Name of the Client KeyPair Secret\ntype: kubernetes.io/tls"
    },
    "serverKeypairSecretName": {
      "type": "string",
      "description": "Name of the Server KeyPair Secret\ntype: kubernetes.io/tls"
    },
    "truststoreCaSecretName": {
      "type": "string",
      "description": "Name of the Truststore Certificates Secret\ntype: Opaque"
    }
  }
}