Task Runners
Available on: Enterprise Edition>= 0.18.0
Task Runner capabilities and supported plugins.
Task Runners offer a powerful way to offload compute-intensive tasks to remote environments.
Task Runner Types
There are a number of task runner types. The Docker and Process task runners are included in the Open Source edition. All other types require an Enterprise Edition license or a Kestra Cloud account.
Enterprise Edition Task Runners:
Task Runners vs Worker Groups
Task Runners and Worker Groups both offload compute-intensive tasks to dedicated workers. However, worker groups have a broader scope, applying to all tasks in Kestra, whereas task runners are limited to scripting tasks (Python, R, JavaScript, Shell, dbt, etc. — see the full list here). Worker groups can be used with any plugins.
For instance, if you need to query an on-premise SQL Server database running on a different server than Kestra, your SQL Server Query task can target a worker with access to that server. Additionally, worker groups can fulfill the same use case as task runners by distributing the load of scripting tasks to dedicated workers with the necessary resources and dependencies (incl. hardware, region, network, operating system).
You can read more about the differences on the dedicated page.
Was this page helpful?