​Centralized ​Task ​Configuration

How to centrally govern your task configuration in a modular way.

The Namespace page allows you to configure secrets, task defaults and variables that can be used within any flow in that namespace.

Secrets

On the Namespaces page, select the namespace where you want to define the secrets and go to the Secrets tab. Here, you will see all existing secrets associated with this namespace. Click on "Add a secret" button on the top right corner of the page.

You can now define the secret by providing the Key and the Secret value. Save the secret by clicking on the "Save" button at the bottom.

The secret key should now start appearing on the Secrets tab. You can edit the secret's value or delete the secret by clicking on the appropriate button towards the right of the secret row. You can reference the secret in the flow by using the key e.g. "{{ secret('MYSQL_PASSWORD') }}".

Here is how you can use it in a flow:

id: query-mysql
namespace: io.kestra.tests

tasks:
  id: query
  type: "io.kestra.plugin.jdbc.mysql.Query"
  url: jdbc:mysql://localhost:3306/test
  username: root
  password: "{{ secret('MYSQL_PASSWORD') }}"
  sql: select * from employees
  fetchOne: true

Task Defaults

Task Defaults can also be defined at the namespace level. These task defaults are then applied for all tasks of the corresponding type defined in the flows under the same namespace.

On the Namespaces page, select the namespace where you want to define the task defaults and navigate to the Task defaults tab. You can add the task defaults here and save the changes by clicking on the "Save" button at the bottom of the page.

You can reference secrets and variables defined with the same namespace in the task defaults.

In the example below, you no longer need to add the password property for the MySQL query task as it's defined in your namespace-level taskDefaults:

id: query-mysql
namespace: io.kestra.tests

tasks:
  id: query
  type: "io.kestra.plugin.jdbc.mysql.Query"
  url: jdbc:mysql://localhost:3306/test
  username: root
  sql: select * from employees
  fetchOne: true

Variables

Variables defined at the namespace level can be used in any flow defined under the same namespace using the syntax: {{ namespace.variable_name }}.

On the Namespaces page, select the namespace where you want to define the variables. Go to the Variables tab. You can now define the variables on this page. Save the changes by clicking the "Save" button at the bottom of the page.

Here is an example flow where the namespace variable is used:

id: query-mysql
namespace: io.kestra.tests

tasks:
  id: query
  type: "io.kestra.plugin.jdbc.mysql.Query"
  url: jdbc:mysql://localhost:3306/test
  username: "{{ namespace.mysql_user }}"
  sql: select * from employees
  fetchOne: true