​Kubernetes

Install Kestra in a Kubernetes cluster using a Helm chart.

Helm Chart repository

We recommend Kubernetes deployment for production workloads, as it allows you to scale specific Kestra services as needed.

We provide an official Helm Chart to make the deployment easier.

• The chart repository is available under helm.kestra.io.
• The source code of the charts can be found in the kestra-io/helm-charts repository on GitHub.

Install the chart

helm repo add kestra https://helm.kestra.io/
helm install kestra kestra/kestra

By default, the chart will only deploy one standalone Kestra service with one replica. This means that all Kestra server components will be deployed within a single pod. You can change that default behavior and deploy each service independently using the following Helm chart values:

deployments:
  webserver:
    enabled: true
  executor:
    enabled: true
  indexer:
    enabled: true
  scheduler:
    enabled: true
  worker:
    enabled: true
  standalone:
    enabled: false

The chart can also deploy the following related services:

• A Kafka cluster and Zookeeper using kafka.enabled: true
• An Elasticsearch cluster using elasticsearch.enabled: true
• A MinIO standalone using minio.enabled: true
• A PostgreSQL using postgresql.enabled: true

The MinIO (as the internal storage backend) and PostgreSQL (as the database backend) services are enabled by default to provide a fully working setup out of the box.

Configuration

Here is how you can adjust the Kestra configuration:

• Using a Kubernetes ConfigMap via the configuration Helm value.
• Using a Kubernetes Secret via the secrets Helm value.

Both must be valid YAML that will be merged as the Kestra configuration file.

Here is an example showing how to enable Kafka as the queue implementation and configure its bootstrap.servers property using a secret:

configuration:
  kestra:
    queue:
      type: kafka

secrets:
  kestra:
    kafka:
      client:
        properties:
          bootstrap.servers: "localhost:9092"

Docker in Docker (DinD) Worker side car

By default, Docker in Docker (DinD) is installed on the worker in the rootless version. This can be restricted on some environment due to security limitations.

Some solutions you may try:

• On Google Kubernetes Engine (GKE), use a node pool based on UBUNTU_CONTAINERD that works well with Docker DinD, even rootless
• Some Kubernetes clusters support only a root version of DinD; to make your Kestra deployment work, disable the rootless version using the following Helm chart values:

dind:
  image:
    image: docker
    tag: dind
  args:
    - --log-level=fatal