How to Store Component Credentials in a Kubernetes Secret

This guide shows you how to keep the credentials of Platform Manager and the other Axual components out of the values file: write them as a secrets.yml file, store it in a Kubernetes Secret, point the component at that Secret, and replace any credentials that were in Git before.

Type

How-to guide

Goal

Run an Axual component with its credentials read from a Kubernetes Secret, so the values file in Git holds none.

Audience

Platform Operator who can create Secrets and install or upgrade Helm releases in the target namespace.

When to use

Use this guide when you prepare a new installation, or when a security review flags plaintext credentials in an existing values file.

Contents

The sections below cover each task in this guide:

Components that read a credentials Secret

Each component below reads a Secret holding one key, secrets.yml, named by its existingSecretName value. The procedure is the same for all of them; only the values key and the credential keys differ. Platform Manager holds most of the credentials. For the others, a chart that generates keystores also sets their passwords, so their lists hold only the keys you add yourself.

Component Values key Release, namespace Credential keys

Platform Manager

platform-manager.existingSecretName

governance, axual

Credentials Secret

API Gateway

api-gateway.existingSecretName

governance, axual

Credentials Secret

Topic Browse

topic-browse.existingSecretName

governance, axual

None; Topic Browse receives the Kafka and Apicurio Registry credentials per request

Metrics Exposer

metrics-exposer.existingSecretName

governance, axual

Credentials Secret

Rest Proxy

rest-proxy.existingSecretName

streaming, kafka

Credentials Secret

Apicurio Registry Auth Proxy

apicurio-registry-v3.authProxy.existingSecretName

streaming, kafka

Auth Proxy secrets

The release names and namespaces are the ones Install Axual Governance and Install Axual Streaming use. Substitute your own throughout.

Axual Connect and Kafka Connect read individual Secret keys instead of a secrets.yml file. See How to Deploy Axual Connect, How to Set Up Vault for Axual Connect and How to Deploy a Kafka Connect Cluster.

Prerequisites

Confirm the following before you begin.

Access and permissions required

You need the following access and permissions:

  • Permission to create and update Secrets in the component’s namespace.

  • Permission to install or upgrade the Helm release that deploys the component.

  • The value of every credential the component needs, for example the database username and password and the Vault Role ID and Secret ID pairs for Platform Manager.

Tools and versions required

You need the following tools:

  • helm >= 3.12, with OCI registry support.

  • kubectl >= 1.28, configured for the target cluster.

  • jq, for the verification step.

  • A way to create Secrets, such as kubectl, Helm Secrets with Mozilla SOPS, Sealed Secrets or 1Password Secrets. See Production-grade integrations.

Resources that must exist before starting

The following must already exist:

Write the credentials file

The Secret holds one key, secrets.yml, whose value is component configuration in the same structure as the component’s config in values.yaml. The component loads it after config and merges the two. A map entry such as one Tenant-Instance can then keep its URI in config and its password in the Secret.

  1. Write the secrets.yml content, holding only the credential keys. The example below is for Platform Manager and covers the database, the governance Vault and Axual Connect; leave out the blocks for features you don’t use. For another component, use the keys from its row in Components that read a credentials Secret.

    Replace every <VALUE> placeholder with your own value before running a command or applying a file.
    secrets.yml
    spring:
  datasource:
    username: "<DB_USERNAME>"
    password: "<DB_PASSWORD>"
governance:
  vault:
    roleId: "<GOVERNANCE_VAULT_ROLE_ID>" (1)
    secretId: "<GOVERNANCE_VAULT_SECRET_ID>"
axual:
  connect:
    instanceConnectCredentials:
      <TENANT>-<INSTANCE>: (2)
        username: "<CONNECT_USERNAME>"
        password: "<CONNECT_PASSWORD>"
connectorVault:
  instances:
    <TENANT>-<INSTANCE>: (2)
      roleId: "<CONNECT_VAULT_ROLE_ID>"
      secretId: "<CONNECT_VAULT_SECRET_ID>"
    1 Write roleId and secretId in camelCase under governance.vault. With role-id and secret-id, Platform Manager fails to start with Could not resolve placeholder 'governance.vault.roleId'.
    2 Use the same <TENANT>-<INSTANCE> key as the matching entry in config, for example axual-dev. Add one entry per Tenant-Instance.

  2. Add every other credential the component’s config sets, such as the Keycloak admin password or the SMTP password for Platform Manager. The Credential keys column in Components that read a credentials Secret links to each component’s list.

  3. Keep the credential keys out of the component’s config in values.yaml. Keep the non-secret keys, such as spring.datasource.url, governance.vault.uri and connectorVault.instances.<TENANT>-<INSTANCE>.uri for Platform Manager, in config.

    A value in the Secret overrides the same key in config. A credential left in config stays in Git.

  4. Leave the component’s secrets block out of values.yaml. The component doesn’t read it once existingSecretName is set, and its values would stay in Git.

Create the Secret

Create the Secret in the component’s namespace, with the tool you already use for Secrets. Whichever tool you use, the result has this shape:

apiVersion: v1
kind: Secret
metadata:
  name: platform-manager-credentials
  namespace: axual
type: Opaque
stringData:
  secrets.yml: | (1)
    spring:
      datasource:
        username: "<DB_USERNAME>"
        password: "<DB_PASSWORD>"
1 The key must be named secrets.yml, and its value is the multi-line content from Write the credentials file.

Don’t commit this manifest unencrypted.

Point the component at the Secret

Set existingSecretName to the Secret’s name, then install or upgrade the release.

  1. Add the Secret name to your values.yaml, under the values key from Components that read a credentials Secret.

    values.yaml
    platform-manager:
  existingSecretName: "platform-manager-credentials"

  2. Install or upgrade the release with the updated values. For a new installation, run the install command from Install Axual Governance or Install Axual Streaming. For an existing release, upgrade it:

    helm upgrade governance oci://registry.axual.io/axual-charts/axual-governance \
  --version <GOVERNANCE_CHART_VERSION> -f ./values.yaml -n axual

    The upgrade is safe to re-run. To go back to the previous configuration, run helm rollback governance -n axual. A rollback restores the credentials the previous values held.

Verify the component reads the Secret

Confirm the component mounts your Secret, starts with it, and that the release values hold no credential for it.

  1. List the Secrets each Deployment in the namespace mounts.

    kubectl get deployments -n axual \
  -o custom-columns='NAME:.metadata.name,SECRETS:.spec.template.spec.volumes[*].secret.secretName'

    The component’s Deployment lists your Secret, for example platform-manager-credentials.

  2. Check that the component’s pod is ready.

    kubectl get pods -n axual

    The component’s pod reads 1/1 and Running. A pod that restarts with a login or connection error in its logs has a wrong or missing key in secrets.yml. Check the key names against the component’s credential list, then update the Secret.

  3. Check that the component’s values in the release hold no credential. Replace platform-manager with the component’s values key. The command prints key paths only, never values.

    helm get values governance -n axual -o json | jq -c '."platform-manager"
  | [paths(scalars) | map(tostring) | join(".")
  | select(test("(password|secretid|secret-id|salt)$"; "i"))]'

    The command prints []. Any path it prints is a credential still in values.yaml; move it to secrets.yml.

Replace the credentials that were in Git

This section applies only when the credentials were in values.yaml before. Moving a credential out of values.yaml doesn’t remove it from Git history. Helm also keeps every previous release’s values, which helm get values governance -n axual --revision <REVISION> prints.

Treat every credential that was ever committed as exposed, and replace each one. For Platform Manager, generate a new Vault Secret ID as described in How to Set Up Vault for Governance and How to Set Up Vault for Axual Connect, and change the database and Axual Connect passwords. Then apply the new values as in Change a credential later.

Change a credential later

The component reads the Secret at startup, so a changed value applies only after a restart. The Secret is replaced as a whole, not merged.

  1. Update the Secret with the full secrets.yml content, including the credentials that don’t change. A key missing from the new content disappears from the Secret.

  2. Restart the component’s Deployment.

    kubectl rollout restart deployment/<DEPLOYMENT_NAME> -n axual

    Take <DEPLOYMENT_NAME> from the first command in Verify the component reads the Secret.