Offload and Isolate Task Execution with Kestra Task Runners
For the complete documentation index, see llms.txt. For a full content snapshot, see llms-full.txt. Append.mdto anykestra.io/docs/*URL for plain Markdown.
Available on:Open Source EditionEnterprise Edition
Task runners decide where a script task’s code runs: on the worker host, in a container, or on remote compute.
Offload and isolate compute with task runners
Many data processing tasks are computationally intensive and require a lot of resources (such as CPU, GPU, and memory). Instead of provisioning always-on servers, Task Runners can execute your code on dynamically provisioned compute instances in the cloud, such as AWS Batch, Azure Batch, Google Batch, auto-scaled Kubernetes clusters, and more.
All you have to do to offload your task execution to a remote environment is to specify the taskRunner type in your task configuration. Each type of a task runner is a plugin with its own schema. The built-in code editor provides documentation, autocompletion, and syntax validation for all task runner plugin properties to ensure correctness, standardization, and consistency.
Some task runner plugins are available only in the Enterprise Edition. If you want to try them out, please reach out. See Open Source vs Enterprise to compare editions.
Task Runner Overview
Learn about Kestra Task Runners capabilities and supported plugins for executing tasks in diverse environments.
Task Runner Benefits
Explore the benefits of using Task Runners in Kestra for isolated execution, resource control, and deployment flexibility.
Task Runner vs. Worker Group
Learn when to use Task Runners versus Worker Groups in Kestra for optimal compute resource management and isolation.
Task Runner Types
Explore the different types of Task Runners available in Kestra, including Docker, Process, Kubernetes, and cloud-based runners.
Was this page helpful?