​Process ​Task ​Runner

Run tasks as local processes.

How to use the Process task runner

Here is an example of a Shell script configured with the Process task runner which runs a Shell command as a child process in the Kestra host:

id: process_script_runner
namespace: company.team

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - echo "Hello World!"

The Process task runner doesn’t have any additional configuration beyond the type property.

Benefits

The Process task runner is useful if you want to access local files e.g. to take advantage of locally configured software libraries and virtual environments.

Combining task runners with Worker Groups

You can combine the Process task runner with Worker Groups to run tasks on dedicated servers that might have specific software libraries or configurations. This powerful combination allows you to leverage the compute resources of your Worker Groups while running tasks as local processes without the overhead of containerization.

The example below shows how to combine the Process task runner with Worker Groups to fully leverage the GPU resources of a dedicated server:

id: python_on_gpu
namespace: company.team

tasks:
  - id: gpu_intensive_ai_workload
    type: io.kestra.plugin.scripts.python.Commands
    namespaceFiles:
      enabled: true
    commands:
      - python main.py
    workerGroup:
      key: gpu
    taskRunner:
      type: io.kestra.plugin.core.runner.Process