​Task ​Runners

Task Runners is an extensible, pluggable system capable of executing your tasks in arbitrary remote environments.

Many data processing tasks are computationally intensive and require a lot of resources (such as CPU, GPU, and memory). Instead of provisioning always-on servers, Task Runners can execute your code on dynamically provisioned compute instances in the cloud, such as AWS ECS Fargate, Azure Batch, Google Batch, auto-scaled Kubernetes clusters, and more.

All you have to do to offload your task execution to a remote environment is to specify the taskRunner type in your task configuration. Each type of a task runner is a plugin with its own schema. The built-in code editor provides documentation, autocompletion, and syntax validation for all task runner plugin properties to ensure correctness, standardization, and consistency.

Task Runner Overview

Understand the capabilities of Task Runners and the plugins that support them.

Task Runner Benefits

Discover how Task Runners simplify resource allocation, environment management, and deployment across environments.

Task Runner vs. Worker Group

Find out when to use Task Runners or Worker Groups.

Task Runner Types

This section lists all task runners available in Kestra.