Environment-Aware Secret Fetching with Dynamic Templating

Hardcoding one flow per environment triples your maintenance surface; hand-rolling secret retrieval in every script triples it again. This Kestra blueprint solves both at once. A SELECT input chooses the environment, a Pebble expression builds the secret key at runtime ({{ secret(inputs.environment | upper ~ '_DB_PASSWORD') }}), and the resolved value lands in the script as a plain environment variable, auto-redacted from execution logs. The inline script doubles as documentation: it shows, side by side, the IAM-role-and-two-API-calls dance you skip by letting the orchestrator own secret resolution.

How it works

1. The environment input (type: SELECT) offers dev, staging, or prod, defaulting to dev.
2. The fetch_secret_the_kestra_way task (io.kestra.plugin.scripts.shell.Commands) declares env.DB_PASSWORD as {{ secret(inputs.environment | upper ~ '_DB_PASSWORD') }}. Pebble uppercases the input and concatenates the suffix, so choosing staging resolves the STAGING_DB_PASSWORD secret at execution time.
3. The comparison script ships inline via inputFiles, so the blueprint deploys standalone with no namespace files or repo checkout. It verifies DB_PASSWORD arrived, then "connects" using it. Kestra masks any secret() value that would appear in the logs.

What you get

• One flow definition for every environment, no forked dev/staging/prod copies drifting apart.
• Secret names computed at runtime with Pebble filters and concatenation, not hardcoded per flow.
• Zero secret-retrieval boilerplate in your scripts: no sts assume-role, no SSM or Secrets Manager calls, no jq plumbing.
• Automatic redaction of resolved secret values in the execution log UI.

Who it's for

• Data and platform engineers maintaining near-identical flows across environments.
• Security-conscious teams centralizing secret access in the orchestrator instead of scattering IAM logic across scripts.
• Developers migrating bash pipelines whose first 40 lines are credential plumbing.

Why orchestrate this with Kestra

Every script that fetches its own secrets re-implements auth, error handling, and redaction, in every language your team uses. Kestra moves that concern into the platform: the secret() function resolves values from the configured secret backend at runtime, templating makes the lookup dynamic per input or environment, and injection happens through task env, so scripts stay portable and testable. Combined with inputs, one YAML definition covers your whole promotion path, and every execution records exactly which environment it ran against.

Prerequisites

• A Kestra instance with a secret backend configured (built-in secrets, or Vault, AWS Secrets Manager, and others on the Enterprise Edition).
• Ability to run shell script tasks on the worker.

Secrets

• DEV_DB_PASSWORD, STAGING_DB_PASSWORD, PROD_DB_PASSWORD: the environment-specific database passwords. Only the one matching your chosen environment input needs to exist for a given run.

Quick start

1. Create the DEV_DB_PASSWORD secret (any value works for the demo).
2. Import this flow into the company.team namespace.
3. Execute it with environment: dev and open the logs: the script confirms the secret arrived and the value is masked.
4. Add the staging and prod secrets, then rerun with the other environments; nothing in the flow changes.

How to extend

• Replace the demo script with a real client (psql, mysql, an API call) that consumes DB_PASSWORD.
• Derive more values from the same input, for example DB_HOST: "{{ secret(inputs.environment | upper ~ '_DB_HOST') }}".
• Move the environment choice from a manual input to trigger context, so schedules in different namespaces pin their own environment via flow defaults.
• Combine with io.kestra.plugin.core.flow.Subflow to pass the environment down through an entire pipeline.

Links

• Secrets documentation: https://kestra.io/docs/concepts/secret
• Pebble expressions: https://kestra.io/docs/expressions