Docker Compose​Docker ​Compose

Start Kestra with a PostgreSQL database backend by using a Docker Compose file.


The quickest way to a production-ready, lightweight Kestra installation is to leverage Docker and Docker Compose. This guide helps you get started with Kestra using Docker.

Before you begin

Make sure you have the following installed:

Download the Docker Compose file

Download the Docker Compose file using the following command on Linux and macOS:

bash
curl -o docker-compose.yml \
https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml

On Windows, use the following command:

powershell
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml" -OutFile "docker-compose.yml"

You can also download the Docker Compose file manually and save it as docker-compose.yml.

Launch Kestra

Use the following command to start the Kestra server:

bash
docker-compose up -d

Open the URL http://localhost:8080 in your browser to launch the UI.

Adjusting the configuration

The command from the previous section starts a standalone server, with all architectural components running in one JVM.

The configuration is done inside the KESTRA_CONFIGURATION environment variable of the Kestra container. You can update the environment variable inside the Docker Compose file or pass it as a Docker CLI argument.

Finally, you can also use the host network mode for the kestra service. This makes your container use your host network, and you are then able to reach all your exposed ports. This means you have to change the services.kestra.environment.KESTRA_CONFIGURATION.datasources.postgres.url to jdbc:postgresql://localhost:5432/kestra. This is the easiest way reach all ports, but it can be a security risk.

See the example below using network_mode: host.

Example

Postgres 16 not compatible with 17 error

By default, the Docker Compose template uses the latest image for PostgreSQL. However, if you initialized your Kestra database on an older version of PostgreSQL, you might encounter the following error:

text
The data directory was initialized by PostgreSQL version 16, which is not compatible with this version 17.0 (Debian 17.0-1.pgdg120+1).

To resolve this, you need to specify a specific tag for the PostgreSQL image in your Docker Compose file. In the example below, we specify 16 as the error above was originally initialized by version 16:

yaml
services:
  postgres:
    image: postgres:16

SIGILL in Java Runtime Environment on MacOS M4

Add the following environment variable to your Kestra container: -e JAVA_OPTS="-XX:UseSVE=0":

bash
docker run --pull=always --rm -it -p 8080:8080 --user=root -e JAVA_OPTS="-XX:UseSVE=0" -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp kestra/kestra:latest server local

To apply the same setting in a Docker Compose file:

yaml
services:
  kestra:
    image: kestra/kestra:latest
    environment:
      JAVA_OPTS: "-XX:UseSVE=0"

Kestra with server components in different services

Server components can run independently from each other. Each of them communicate through the database. The kestra server command starts each server component individually:

  • kestra server executor
  • kestra server worker
  • kestra server indexer
  • kestra server scheduler
  • kestra server webserver

For more details on Kestra server commands, check out the Server CLI documentation.

Here is an example Docker Compose configuration file running Kestra services with replicas on the Postgre database backend.

yaml
volumes:
  postgres-data:
    driver: local
  kestra-data:
    driver: local

services:
  postgres:
    image: postgres
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: kestra
      POSTGRES_USER: kestra
      POSTGRES_PASSWORD: k3str4
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
      interval: 30s
      timeout: 10s
      retries: 10

  kestra-scheduler:
    image: kestra/kestra:latest
    deploy:
      replicas: 2
    pull_policy: if_not_present
    user: "root"
    command: server scheduler
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: &common_configuration |
        datasources:
          postgres:
            url: jdbc:postgresql://postgres:5432/kestra
            driver-class-name: org.postgresql.Driver
            username: kestra
            password: k3str4
        kestra:
          server:
            basic-auth:
              username: "[email protected]"
              password: kestra
          repository:
            type: postgres
          storage:
            type: local
            local:
              base-path: "/app/storage"
          queue:
            type: postgres
          tasks:
            tmp-dir:
              path: /tmp/kestra-wd/tmp
    ports:
      - "8082-8083:8081"
    depends_on:
      postgres:
        condition: service_started

  kestra-worker:
    image: kestra/kestra:latest
    deploy:
      replicas: 2
    pull_policy: if_not_present
    user: "root"
    command: server worker
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: *common_configuration
    ports:
      - "8084-8085:8081"
    depends_on:
      postgres:
        condition: service_started
  kestra-executor:
    image: kestra/kestra:latest
    deploy:
      replicas: 2
    pull_policy: if_not_present
    user: "root"
    command: server executor
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: *common_configuration
    ports:
      - "8086-8087:8081"
    depends_on:
      postgres:
        condition: service_started
  kestra-webserver:
    image: kestra/kestra:latest
    deploy:
      replicas: 1
    pull_policy: if_not_present
    user: "root"
    command: server webserver
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: *common_configuration
      KESTRA_URL: http://localhost:8080/
    ports:
      - "8080:8080"
      - "8081:8081"
    depends_on:
      postgres:
        condition: service_started

Was this page helpful?