Govern cloud infrastructure with Kestra Assets: automated S3 bucket provisioning by team

Provision cloud infrastructure as code and turn raw AWS resources into first-class, governed assets with Kestra. This blueprint creates one Amazon S3 bucket per selected team using the AWS CLI, then registers each bucket as a discoverable AWS_BUCKET asset so downstream workflows can reference it explicitly. It solves the common problem of "ghost" cloud resources: buckets that exist somewhere in an account but have no clear owner, no lineage, and no catalog entry.

How it works

1. A teams input of type MULTISELECT lets the user pick any combination of Business, Data, Finance, and Product.
2. The for_each task (io.kestra.plugin.core.flow.ForEach) iterates over the selected teams.
3. For each team, the create_bucket task (io.kestra.plugin.aws.cli.AwsCLI) runs aws s3 mb s3://kestra-{team}-bucket, slugifying the team name with the slugify filter.
4. The same task declares an assets.outputs entry of type: AWS_BUCKET, attaching metadata for provider (s3) and the canonical address (s3://...), so the bucket is published to the Kestra asset catalog as it is created.
5. pluginDefaults injects AWS credentials and region into every io.kestra.plugin.aws task and sets allowFailure: true so a single failed creation does not abort the run.

What you get

• One S3 bucket per chosen team, named consistently as kestra-<team>-bucket.
• A governed AWS_BUCKET asset for every bucket, complete with provider and address metadata.
• A clear provider/consumer contract: platform teams provision, application and data teams consume.
• Infrastructure that is visible, auditable, and reusable instead of scattered and untracked.

Who it's for

Platform engineers, DevOps and SRE teams, and infrastructure owners who manage shared cloud resources across multiple internal teams and need ownership and lineage, not just provisioning.

Why orchestrate this with Kestra

The AWS CLI and IaC tools create buckets, but they do not give you a living catalog of who owns what or which workflows depend on each resource. Kestra adds event triggers, automatic retries, and full execution lineage on top of declarative YAML, and its asset model captures the one thing the AWS toolchain cannot: the relationship between infrastructure and the workflows that consume it. Buckets become catalog entries that downstream flows reference as typed inputs, giving you governance across teams from a single control plane.

Prerequisites

• A Kestra instance (asset registration requires Enterprise Edition).
• AWS credentials with permission to create S3 buckets.

Secrets

• AWS_ACCESS_KEY
• AWS_SECRET_ACCESS_KEY
• AWS_REGION

Quick start

1. Add the three secrets above to your Kestra instance.
2. Import this blueprint into the kestra.company namespace.
3. Run the flow and select one or more teams in the teams input.
4. Open the asset catalog to see each AWS_BUCKET asset and its metadata.

How to extend

• Add new teams to the teams input values to provision more buckets.
• Enable versioning, encryption, or tagging by extending the aws s3 commands.
• Build a downstream flow (for example io.kestra.plugin.aws.s3.Upload) that consumes a bucket as an assets.inputs entry and emits derived file assets, establishing end-to-end lineage.
• Swap the S3 logic for other AWS resources to register databases, queues, or compute as assets too.

Links

• Kestra AWS plugin documentation
• AWS orchestration with Kestra