Concurrency limits
Available on: >= 0.13.0
Control concurrent executions of a given flow.
The flow level concurrency
property allows you to control the number of concurrent executions of a given flow by setting the limit
key.
You can treat concurrency as a global concurrency limit for that specific flow. The concurrency limit and behavior is then applied to all executions of that flow, regardless of whether those executions have been started automatically via a trigger, webhook, an API call or manually created from the UI.
For example, if you set the concurrency limit
to 2, only two executions of that flow will be allowed to run at the same time. If you try to trigger a third execution, it will be queued until one of the two running executions is completed.
id: concurrency_example
namespace: company.team
concurrency:
limit: 2
tasks:
- id: wait
type: io.kestra.plugin.scripts.shell.Commands
commands:
- sleep 10
As you can see in the UI, the third execution has been queued while the first two have finished successfully.
behavior
property
You can customize the execution behavior to CANCEL or FAIL an execution if the concurrency limit is reached. To do that, set the behavior
Enum-type property to one of the following values:
QUEUE
CANCEL
FAIL
.
Let's say you set the concurrency.limit
to 2, and you use the CANCEL
or FAIL
behavior. The third execution's state will be immediately set to CANCELLED
or FAILED
status respectively without running any task.
Here is a full flow example that uses the concurrency
property to limit the number of concurrent executions to 2. The bash
task will sleep for 10 seconds, so you can trigger multiple executions of that flow and see how the concurrency
property behaves.
id: concurrency_limited_flow
namespace: company.team
concurrency:
behavior: FAIL # QUEUE, CANCEL or FAIL
limit: 2 # can be any integer >= 1
tasks:
- id: wait
type: io.kestra.plugin.scripts.shell.Commands
taskRunner:
type: io.kestra.plugin.core.runner.Process
commands:
- sleep 10
As you can see in the UI, the third execution failed as the first two executions were still running.
Was this page helpful?