Developer Guide
Develop Python, R, Shell, PowerShell, Julia, Ruby or Node.js scripts and integrate them with Git and CI/CD.
Data storage and processing
Kestra's primary purpose is to orchestrate data processing via tasks, so data is central to each flow's execution.
Errors and retries
Errors are special branches of a flow where you can define how to react to task failures (e.g. send an email, open a ticket, etc.).
Namespace Files
This page covers the concept of Namespace Files and how to use them in your flows.
Version Control with Git
Kestra supports version control with Git. You can use one or more Git repositories to store your Flows and Namespace Files, and track changes to them over time via Git commit history.
Caching
Kestra provides a file caching, which is especially useful when you work with sizeable package dependencies that don't change often.
Build a Custom Plugin
Browse Kestra's integrations and learn how to create your own plugins.
Programming Languages
Kestra currently supports the following programming languages: Python, R, Node.js, Shell, Powershell, Julia and Ruby.
Commands and Script tasks
For each of the supported languages (a.o. Python, R, Node.js, Shell), Kestra provides two types of tasks: Script and Commands.
Inline Scripts in Docker
To get started with a Script task, paste your custom script inline in your YAML workflow definition along with any other configuration.
DOCKER and PROCESS runners
Kestra supports two runners for scripting tasks: DOCKER and PROCESS.
Building a custom Docker image
This page shows how to build a custom Docker image for your script tasks.
Installing dependencies at runtime
There are several ways of installing custom packages for your workflows. This page shows how to install dependencies at runtime using the beforeCommands property.
Additional languages (Rust)
Kestra is language agnostic. This section shows how to use Rust in your flows.
Bind mount
To run a script stored locally, you can bind-mount it to your Kestra container.
The Git Clone task
This task allows you to clone a Git repository into a working directory and then use the files from that repository in downstream tasks.
Python, R, Node.js and Shell Scripts
Kestra is language agnostic. You can orchestrate custom business logic written in Python, R, Julia, Ruby, Node.js, Powershell and Shell scripts, and you can also build custom plugins in Java.
Output directory
If you want to generate files in your script to make them available for download and use in downstream tasks, you can leverage either the outputFiles property or the outputDir expression.
Outputs and metrics
Your scripts can send outputs and metrics to Kestra's backend during flow execution. This allows you to track custom metadata and visualize it across multiple executions of a flow.
Working Directory
This task allows you to run multiple tasks sequentially in the same working directory. It is useful when you want to share files from Namespace Files or from a Git repository across multiple tasks.
GitHub Actions
We provide two official GitHub Actions to help you create a CI/CD pipeline for your Kestra flows.
GitLab CI
This page describes how to use GitLab CI to create a CI/CD pipeline for your Kestra flows.
Terraform
This page describes how to use Terraform to provision and manage changes to Kestra resources.
Helpers
Kestra provides some helper functions that can help during flow development.
Azure DevOps
This page describes how to use Azure DevOps to create a CI/CD pipeline for your Kestra flows.
BitBucket Pipes
This page describes how to use BitBucket Pipes to create a CI/CD pipeline for your Kestra flows.
Create a CI/CD pipeline
Kestra provides several ways to create a CI/CD pipeline for your flows. This section explains how to automate the validation and deployment of your workflows using CI/CD.