docstutorial

Scripts​Scripts

Kestra is language agnostic - Run custom scripts.

Run custom Python, R, Julia, Node.js, Shell scripts and more in isolated Docker containers.

Tasks running in Docker containers

Many tasks in Kestra will run in dedicated Docker containers, including among others:

Kestra uses Docker for those tasks to ensure that they run in a consistent environment and to avoid dependency conflicts.

Defining a Docker task runner

To run a task in a Docker container, set the taskRunner property with the type io.kestra.plugin.scripts.runner.docker.Docker:

yaml
taskRunner:
  type: io.kestra.plugin.scripts.runner.docker.Docker

Many tasks, including Python, use the io.kestra.plugin.scripts.runner.docker.Docker task runner by default.

Using the containerImage property, you can define the Docker image for the task. You can use any image from a public or private container registry, as well as a custom local image built from a Dockerfile. You may even build a Docker image using the Docker plugin in one task and reference the built image by the tag in a downstream task.

yaml
containerImage: ghcr.io/kestra-io/pydata:latest

The taskRunner property also allows you to configure credentials with nested username and password properties to authenticate a private container registry.

yaml
id: private_docker_image
namespace: company.team

tasks:
  - id: custom_image
    type: io.kestra.plugin.scripts.python.Script
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
      credentials:
        username: your_username
        password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    containerImage: ghcr.io/your_org/your_package:tag
    script: |
        print("this runs using a private container image")

The task documentation of each script task (e.g., Python, R, Julia, Node.js, and Shell) provides more details about available taskRunner properties.

Run a script without a language plugin

You can also use the Docker Task Runner to run scripts of a programming language that does not have a plugin built in to Kestra like Python, Julia, and the others above. Below we use a combination of Namespace files and the Docker task to run a simple script written in Go.

Our flow looks as follows with the container image specifying the latest image for Go, golang:latest. Also make sure to enable Namespace files with enabled: true.

yaml
id: golang
namespace: tutorial

tasks:
  - id: go
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
    containerImage: golang:latest
    namespaceFiles:
      enabled: true
    commands:
      - go run scripts/hello_world.go

Next, we need to make sure the scripts/hello_world.go path exists in our Namespace file directory. After saving your flow, click on Files and add the following file into a folder called scripts:

go
package main

import "fmt"

func main() {
    fmt.Println("Hello, World!")
}

Once saved, click Execute, and you can see the "Hello World" message in the execution logs.

go-script

For more examples of running scripts in Docker containers, check the Script tasks page.

Run a script task with Process task runner

Alternatively to executing commands in a Docker container, you can run a Shell command as a child process in the Kestra host with the Process task runner. The example flow below demonstrates running a simple "Hello World" Shell script specifying the taskRunner as type: io.kestra.plugin.core.runner.Process:

yaml
id: process_script_runner
namespace: company.team

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - echo "Hello World!"

The Process task runner is useful if you want to access local files, for example, to take advantage of locally configured software libraries and virtual environments. For more details, check out the Process task runner documentation

Next steps

Congrats! 🎉 You've completed the tutorial.

Next, you can dive into:

Was this page helpful?