Core Plugins and tasks Parallel

Core Plugins and tasks Parallel

Certified

Run child tasks in parallel.

Starts all child tasks concurrently, bounded by concurrent if set (0 = no cap). Each branch can contain its own sequences or nested flows.

Use when independent steps can run at the same time to shorten wall-clock duration.

yaml
type: io.kestra.plugin.core.flow.Parallel

Run tasks in parallel

yaml
id: parallel
namespace: company.team

tasks:
  - id: parallel
    type: io.kestra.plugin.core.flow.Parallel
    tasks:
      - id: 1st
        type: io.kestra.plugin.core.debug.Return
        format: "{{ task.id }} > {{ taskrun.startDate }}"

      - id: 2nd
        type: io.kestra.plugin.core.debug.Return
        format: "{{ task.id }} > {{ taskrun.id }}"

  - id: last
    type: io.kestra.plugin.core.debug.Return
    format: "{{ task.id }} > {{ taskrun.startDate }}"

Run two sequences in parallel

yaml
id: parallel_sequences
namespace: company.team

tasks:
  - id: parallel
    type: io.kestra.plugin.core.flow.Parallel
    tasks:
      - id: sequence1
        type: io.kestra.plugin.core.flow.Sequential
        tasks:
          - id: task1
            type: io.kestra.plugin.core.debug.Return
            format: "{{ task.id }}"

          - id: task2
            type: io.kestra.plugin.core.debug.Return
            format: "{{ task.id }}"

      - id: sequence2
        type: io.kestra.plugin.core.flow.Sequential
        tasks:
          - id: task3
            type: io.kestra.plugin.core.debug.Return
            format: "{{ task.id }}"

          - id: task4
            type: io.kestra.plugin.core.debug.Return
            format: "{{ task.id }}"

Stop the other branches as soon as one task fails

yaml
id: parallel_fail_fast
namespace: company.team

tasks:
  - id: parallel
    type: io.kestra.plugin.core.flow.Parallel
    onChildFailure: CANCELLED
    tasks:
      - id: fails_fast
        type: io.kestra.plugin.core.execution.Fail

      - id: long_running
        type: io.kestra.plugin.core.flow.Sleep
        duration: PT1M
Properties
Min items1
Default0

Number of concurrent parallel tasks that can be running at any point in time

If the value is 0, no limit exist and all tasks will start at the same time.

List of tasks to run if any tasks failed on this FlowableTask.

DefaultFAIL
Possible Values
CONTINUECANCELFAIL

What to do with the other still-running tasks when one task fails.

CONTINUE (default): other tasks keep running to completion, as today.

CANCELLED / FAILED: as soon as a task fails with no retry left, every other still-running task in this Parallel is interrupted and lands in the given state. The Parallel itself still resolves to FAILED and its errors/finally tasks still run normally.

More Plugins in this Category

Ruby
Ruby

Tasks that run Ruby scripts.

Core

4 tasks

Lua
Lua

Tasks that run Lua scripts.

Core

2 tasks