​Read-only ​Secrets

Integrate external secrets managers in a read-only mode.

Read-only secrets

When integrating an external secrets manager with Kestra, you may want to ensure that those secrets cannot be modified within Kestra, maintaining immutability. Currently, read-only secrets can be configured for AWS Secret Manager, Azure Key Vault, Google Secret Manager, and Vault.

Configure read-only secrets

Read-only secrets can be configured globally in the configuration file as well as enabled from the UI at the Tenant and the Namespace level.

To enable for a specific Tenant, toggle the setting on in the Dedicated secrets manager configuration.

To enable for a specific Namespace, toggle the setting on in the Dedicated secrets manager configuration of the Edit tab.

Secrets will display a lock icon to indicate read-only status, and the Create New Secret button will no longer be visible.

To configure globally, add readOnly: true to the configuration of your external secret manager like in the examples below.

AWS Secret Manager

For compatibility with Kestra, ensure that your AWS secrets are stored as plain text in AWS Secrets Manager and not as key-value pairs. The following example shows the configuration for AWS Secret Manager with a read-only secrets backend:

kestra:
  secret:
    type: aws-secret-manager
    readOnly: true
    awsSecretManager:
      accessKeyId: mysuperaccesskey
      secretKeyId: mysupersecretkey
      region: us-east-1

Azure Key Vault

The following example shows the configuration for Azure Key Vault with a read-only secrets backend:

kestra:
  secret:
    type: azure-key-vault
    readOnly: true
    azureKeyVault:
      clientSecret:
        tenantId: "id"
        clientId: "id"
        clientSecret: "secret"

Google Secret Manager

The following example shows the configuration for Google Secret Manager with a read-only secrets backend:

kestra:
  secret:
    type: google-secret-manager
    readOnly: true
    googleSecretManager:
      project: gcp-project-id
      serviceAccount: |
        Paste the contents of the service account JSON key file here.

Vault

With Vault, secrets are stored in a unique structure that can vary depending on the organization and version of Vault. Typically, there is a Secret Engine that hosts different Secrets with specific paths. Those Secrets are the paths to subkeys that are the actual key value pairs such as Username or Password to a service (e.g., MY_SECRET = MY_SECRET_PASSWORD).

Here’s an example directory structure of a Vault secret engine used with Kestra:

secret/
  ├── app1/
  │   ├── db/ <-- SECRET
  │   │   ├── DATABASE_USERNAME    # Subkey
  │   │   ├── DATABASE_PASSWORD   # Subkey
  │   ├── api/ <-- SECRET
  │       ├── keys                # Subkey
  │       ├── API_TOKEN           # Subkey
  ├── app2/
      ├── config

• secret: This is the secret engine.
• app1 and app2: These are the path names to the secrets. This could be for example separate business units or applications.
• db, api, and config: These are the secret names visible in the Kestra UI. api could be the Vault Secret that contains all API Keys for an application's external services.
• DATABASE_USERNAME, DATABASE_PASSWORD, keys, API_TOKEN: These are the subkey key value pairs that can be used in a Kestra flow.

To configure access to secrets under app1, use the following Kestra configuration with the added property secretPathPrefix:

address: https://my-vault:8200/
rootEngine: secret
secretPathPrefix: app1
token:
  token: my-vault-access-token

This configuration gives Kestra access to the db and api secrets, as they are the secrets on the app1 path. In a flow, to access the value for the subkey API_TOKEN, you write the secret() function with the specified parameters {{ secret('api', subkey='API_TOKEN') }}.

Vault full example

The following steps are a full example of configuring Vault as your secret manager with read-only secrets enabled. This example uses KV Secrets Engine - Version 2 with Vault Enterprise, so rootEngine and namespace are used as optional properties.

In Vault, we have a Secrets Engine named business-unit in the admin namespace that hosts the path to our database password that we want to use to add a table and populate with data in Neon.

In Kestra, we can now navigate to the Namespace we want to set up Vault as a secrets manager for and enter the configuration details:

After saving, we can move to the Secrets tab and see which paths we have access to. Notice the lock icon indicating that read-only is successfully turned on. No new secrets can be created from Kestra, and existing secrets are not editable.

In Vault, we know my-app is the secret that hosts the subkey we are looking for, in this case, NEON_PASSWORD.

Now to use in our flow, we need to use the secret() function with the name of our secret my-app and the subkey parameter set to the key of the secret value we want to use, which in this case is NEON_PASSWORD.

After saving the flow and executing, we can see that Kestra successfully accessed the correct value from Vault and added 100 rows to our Neon database.