​Task ​Runner ​Benefits

Discover how Task Runners simplify resource allocation, environment management, and deployment across environments.

Docker in development, Kubernetes in production

Many Kestra users develop their scripts locally using Docker containers and deploy the same code in production as Kubernetes pods.
Thanks to the taskRunner property, switching between environments is seamless.

Below is an example showing how you can combine pluginDefaults with the taskRunner property to use Docker during development and Kubernetes in production — without changing your code.

1. Development environment (namespace / tenant / instance)

pluginDefaults:
  - type: io.kestra.plugin.scripts
    values:
      taskRunner:
        type: io.kestra.plugin.scripts.runner.docker.Docker
        pullPolicy: IF_NOT_PRESENT # In dev, only pull the image when needed
        cpu:
          cpus: 1
        memory:
          memory: 512Mi

2. Production environment (namespace / tenant / instance)

pluginDefaults:
  - type: io.kestra.plugin.scripts
    values:
      taskRunner:
        type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
        namespace: company.team
        pullPolicy: ALWAYS # Always pull the latest image in production
        config:
          username: "{{ secret('K8S_USERNAME') }}"
          masterUrl: "{{ secret('K8S_MASTER_URL') }}"
          caCert: "{{ secret('K8S_CA_CERT') }}"
          clientCert: "{{ secret('K8S_CLIENT_CERT') }}"
          clientKey: "{{ secret('K8S_CLIENT_KEY') }}"
        resources: # Can be overridden by a specific task if needed
          request:
            cpu: "500m"    # Request 1/2 CPU (500 milliCPU)
            memory: "256Mi" # Request 256 MB of memory

Centralized configuration management

The combination of pluginDefaults and taskRunner enables centralized management of your task runner configuration.
For example, you can define AWS credentials at the namespace level for the Batch task runner plugin:

pluginDefaults:
  - type: io.kestra.plugin.ee.aws.runner.Batch
    values:
      accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
      secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
      region: "us-east-1"

This approach ensures consistency and eliminates repetitive configuration across multiple workflows.

Documentation and autocompletion

Each task runner is a self-contained plugin with its own icon, documentation, and property schema.
The built-in Kestra code editor provides autocompletion, inline documentation, and syntax validation for all runner properties.
Clicking on the runner’s name in the editor opens its documentation sidebar for quick reference.

Full customization: build your own Task Runner

For advanced use cases, you can create a custom task runner plugin tailored to your environment.
Simply build it as a JAR file and add it to the plugins directory. Once Kestra restarts, your custom runner will appear as an available option in any script task.