Run multiple subflows in parallel and wait for their completion

Run several independent child flows at the same time instead of one after another, then wait for all of them to finish before the parent continues. This blueprint fans out to three subflows using the core io.kestra.plugin.core.flow.Parallel task, cutting total wall-clock time when the children have no dependencies on each other, while still giving you a single parent run to monitor, retry, and observe.

How it works

1. The parent flow defines a parallel task of type io.kestra.plugin.core.flow.Parallel.
2. Inside it, three io.kestra.plugin.core.flow.Subflow tasks (flow1, flow2, flow3) are launched concurrently, each pointing at a child flow in the company.team namespace.
3. pluginDefaults applies shared settings to every Subflow task so you do not repeat them: namespace: company.team, wait: true (the parent blocks until each child completes), and transmitFailed: true (a failed child marks the parent run as failed).
4. Because all three subflows run in parallel, the parent finishes as soon as the slowest child returns rather than summing their durations.

What you get

• Concurrent execution of independent subflows from one parent run.
• A single execution to track, with each child shown as its own subflow execution.
• Built-in failure propagation so a broken child surfaces on the parent.
• Less boilerplate thanks to pluginDefaults shared across the subflow tasks.

Who it's for

• Data and platform engineers splitting a pipeline into reusable child flows.
• Teams that want faster end-to-end runs by parallelizing independent steps.
• Anyone standardizing on small, composable flows triggered by a parent orchestrator.

Why orchestrate this with Kestra

Kestra lets you describe parallelism declaratively in YAML, attach event triggers, add retries per task, and inspect full lineage across parent and child executions from one UI. Each subflow keeps its own logs, outputs, and execution history while the parent coordinates timing and failure handling, something a single script's ad hoc threading cannot give you cleanly.

Prerequisites

• A running Kestra instance.
• Three child flows (flow1, flow2, flow3) in the company.team namespace.

Secrets

None. This blueprint uses only core tasks and references no secret() values.

Quick start

1. Create the three child flows below in the company.team namespace.
2. Import this parent flow into the same namespace.
3. Execute the parent and watch the three subflows run in parallel.

First flow:

id: flow1
namespace: company.team

tasks:
  - id: get
    type: io.kestra.plugin.core.debug.Return
    format: hi from {{ flow.id }}

Second flow:

id: flow2
namespace: company.team

tasks:
  - id: get
    type: io.kestra.plugin.core.debug.Return
    format: hi from {{ flow.id }}

Third flow:

id: flow3
namespace: company.team

tasks:
  - id: get
    type: io.kestra.plugin.core.debug.Return
    format: hi from {{ flow.id }}

How to extend

• Add more Subflow tasks under parallel to fan out wider.
• Pass inputs to each subflow and read child outputs back in the parent.
• Set wait: false for fire-and-forget children, or keep transmitFailed: true to fail fast.
• Wrap the parallel block in a Sequential flow to stage groups of parallel work.
• Add a trigger so the parent runs on a schedule or in response to an event.

Links

• Plugin docs: https://kestra.io/plugins/plugin-core