​Docker ​Compose

Start Kestra with a PostgreSQL database backend using a Docker Compose file.

The quickest way to a production-ready lightweight Kestra installation is to leverage Docker and Docker Compose. This guide will help you get started with Kestra using Docker.

Before you begin

Make sure you have already installed:

• Docker
• Docker Compose

Download the Docker Compose file

Download the Docker Compose file using the following command on Linux and macOS:

curl -o docker-compose.yml \
https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml

If you're on Windows, use the following command:

Invoke-WebRequest -Uri "https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml" -OutFile "docker-compose.yml"

You can also download the Docker Compose file manually and save it as docker-compose.yml.

Launch Kestra

Use the following command to start the Kestra server:

docker-compose up -d

Open the URL http://localhost:8080 in your browser to launch the UI.

Adjusting the Configuration

The command above starts a standalone server (all architecture components in one JVM).

The configuration will be done inside the KESTRA_CONFIGURATION environment variable of the Kestra container. You can update the environment variable inside the Docker compose file, or pass it via the Docker command line argument.

Use a configuration file

If you want to use a configuration file instead of the KESTRA_CONFIGURATION environment variable to configure Kestra you can update the default docker-compose.yml.

First, create a configuration file containing the KESTRA_CONFIGURATION environment variable defined in the docker-compose.yml file. You can name it application.yaml.

Then, update kestra service in the docker-compose.yml file to mount this file into the container and make Kestra using it via the --config option:

# [...]
  kestra:
    image: kestra/kestra:latest
    pull_policy: always
    # Note that this is meant for development only. Refer to the documentation for production deployments of Kestra which runs without a root user.
    user: "root"
    command: server standalone --worker-thread=128 --config /etc/config/application.yaml
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
      - $PWD/application.yaml:/etc/config/application.yaml
    ports:
      - "8080:8080"
      - "8081:8081"
    depends_on:
      postgres:
        condition: service_started

Networking in Docker Compose

The default docker-compose file doesn't configure networking for the Kestra containers. This means that you won't be able to access any services exposed via localhost on your local machine (e.g., another Docker container with a mapped port). Your machine and Docker container use a different network. To use a locally exposed service from Kestra container, you can use the host.docker.internal hostname or 172.17.0.1. The host.docker.internal address allows you to reach your host machine's services from Kestra's container.

Alternatively, you can leverage Docker network. By default, your Kestra container will be placed in a default network. You can add your custom services to the docker-compose.yml file provided by Kestra and use the services' alias (keys from services) to reach them. Even better would be if you create a new network e.g. network kestra_net and add your services to it. Then you can add this network to the networks section of the kestra service. With this, you will have access via localhost to all your exposed ports.

The example below shows how you can add iceberg-rest, minio and mc (i.e. Minio client) to your Kestra Docker Compose file.

Finally, you can also use the host network mode for the kestra service. This will make your container use your host network and you will be able to reach all your exposed ports. This means you have to change the services.kestra.environment.KESTRA_CONFIGURATION.datasources.postgres.url to jdbc:postgresql://localhost:5432/kestra. This is the easiest way but it can be a security risk.

See the example below using network_mode: host.

Postgres 16 Not Compatible with 17 Error

By default, the Docker Compose template uses the latest image for postgres. However, if you initated your Kestra database on an older version of Postgres to the latest image, you might encounter the following error:

The data directory was initialized by PostgreSQL version 16, which is not compatible with this version 17.0 (Debian 17.0-1.pgdg120+1).

To resolve this, you need to specify a specific tag for the postgres image in your Docker Compose file. In the example below, we specify 16 as the error above was originally initialized by version 16:

services:
  postgres:
    image: postgres:16

SIGILL in Java Runtime Environment on MacOS M4

Add the following environment variable to your Kestra container: -e JAVA_OPTS="-XX:UseSVE=0":

docker run --pull=always --rm -it -p 8080:8080 --user=root -e JAVA_OPTS="-XX:UseSVE=0" -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp kestra/kestra:latest server local

The same in a Docker Compose file:

services:
  kestra:
    image: kestra/kestra:latest
    environment:
      JAVA_OPTS: "-XX:UseSVE=0"