​Docker

Run custom Python, R, Julia, Node.js and Shell scripts in isolated containers.

Tasks running in Docker containers

Many tasks in Kestra will run in dedicated Docker containers, including among others:

• Script tasks running Python, Node.js, R, Julia, Bash, and more
• Singer tasks running data ingestion syncs
• dbt tasks running dbt jobs

Kestra uses Docker for those tasks to ensure that they run in a consistent environment and to avoid dependency conflicts.

Defining a Docker runner

To run a task in a Docker container, set the runner property to DOCKER:

runner: DOCKER

Many tasks, including Python, use the DOCKER runner by default.

Using the docker property, you can define the Docker image for the task. You can use any image from a public or private container registry, as well as a custom local image built from a Dockerfile. You may even build a Docker image using the Docker plugin in one task, and reference the built image by the tag in a downstream task.

docker:
  image: ghcr.io/kestra-io/pydata:latest

The docker property also allows you to configure credentials with nested username and password properties to authenticate a private container registry.

id: private_docker_image
namespace: dev

tasks:
  - id: custom_image
    type: io.kestra.plugin.scripts.python.Script
    docker:
      image: ghcr.io/your_org/your_package:tag
      credentials:
        username: your_username
        password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    script: |
        print("this runs using a private container image")

The task documentation of each script task provides more details about available docker properties.

For more examples of running scripts in Docker containers, check the Script tasks page.

Next steps

Congrats! 🎉 You've completed the tutorial.

Next, you can dive into:

• Architecture
• Key concepts
• Developer Guide that includes a more detailed explanation of everything covered in this tutorial
• Plugins to integrate with external systems
• Deployments to deploy Kestra to production.