🚀 New! Kestra raises $3 million to grow Learn more

​Deployment on ​Kubernetes

The recommended deployment environment for production workloads is Kubernetes. We provide a Helm Chart in order to deploy your cluster.

  • The chart repository is available here.
  • The source code of the charts can be found here.

Quick Start

bash
helm repo add kestra https://helm.kestra.io/
helm install kestra kestra/kestra

Details

You can change the default behavior and configure your cluster by changing the default values.

By default, the chart will only deploy one Kestra standalone service (all Kestra server components in only one pod) with only one replica.

You can also deploy each server independently, using these values:

yaml
deployments:
  webserver:
    enabled: true
  executor:
    enabled: true
  indexer:
    enabled: true
  scheduler:
    enabled: true
  worker:
    enabled: true
  standalone:
    enabled: false

The chart could also deploy all needed services:

  • A Kafka cluster and Zookeeper using kafka.enabled: true
  • An Elasticsearch cluster using elasticsearch.enabled: true
  • A MinIO standalone using minio.enabled: true
  • A PostgreSQL using postgresql.enabled: true

By default, we enable MinIO & PostgreSQL to have a working version.

Configuration

Configuration of Kestra could be made:

  • In a Kubernetes ConfigMap via the configuration Helm value.
  • In a Kubernetes Secret via the secrets Helm value. Both must be valid YAML that will be merged as the Kestra configuration file.

For example, to enable Kafka as the queue implementation and configure its bootstrap.servers property inside a secret:

yaml
configuration:
  kestra:
    queue:
      type: kafka


secrets:
  kestra:
    kafka:
      client:
        properties:
          bootstrap.servers: "localhost:9092"

Docker in Docker (DinD) Worker side car

By default, we are installing Docker in Docker (DinD) on the worker in the rootless version. This can be restricted on some environment due to security limitations.

Some solutions exists:

  • On Google Kubernetes Engine(GKE), use a node pool based on UBUNTU_CONTAINERD that works well with docker DinD even rootless
  • Some cluster will support only a root version of DinD, disabled the rootless version with below configuration:
yaml
dind:
  image:
    image: docker
    tag: dind
  args:
    - --log-level=fatal