Fast, Scalable, and Declarative Data Management

Create and Deploy all kinds of Data Pipelines with Ease and Speed with Kestra: Your All-in-One Solution for Orchestrating Workflows

A screenshot of the user interface of Kestra's application

Add new workflows in minutes. No ops required.

Finally, no dependency conflicts. It just works!

A YAML sample of code for declarative language and construction of Kestra's flows

API-first design with declarative workflow definition

  • Strong typing and autocompletion: iterate quickly when adding new flows, triggers and tasks. If something is incorrect in your workflow definition, you will know in advance thanks to the built-in syntax validation.
  • Support for data-mesh: thanks to the simple YAML interface, built-in documentation and blueprints, domain experts can build their own workflows, too.
  • Intuitive UI built not only for engineers: when you add workflows directly from the UI, the live-updating topology view helps you see dependencies between processes, systems and tasks.

Iterate faster without the overhead of CI/CD

  • Less time spent babysitting orchestration code: Seamlessly configure your tasks to run sequentially, in parallel or only based on a specific condition or presence of events without having to navigate convoluted orchestration code bases.
  • Intuitive dependency management: Easily manage task dependencies — defining proper execution order is as simple as adding a list of tasks in the order you want them to be executed.
  • Built-in documentation and blueprints: No need to navigate bulky documentation to figure out the right task or trigger properties — they are available right next to your code editor.
Image of execution of a task on Kestra with event or time based triggering
A list of flowable tasks such as sequential, parallel, loop, switch, pause and template

Flexible Event-Driven Workflow Execution

  • Time-based scheduling: Workflows run automatically at specific times, such as daily, weekly, or monthly without the need for manual intervention or inspecting crontabs.
  • Event-based orchestration: Automate workflow executions based on real-time events. Trigger workflows to run automatically in response to events, such as the arrival of new data in a database, file detection in your cloud storage bucket, new message in a message queue, or the completion of other workflows.

Execute and Monitor in Real Time

  • Real-time Workflow Monitoring: Progress tracking of workflows and individual tasks in real-time. Troubleshoot failure by replaying failed tasks or doing a full-text search of the workflow execution logs.
  • Actionable Insights: you can build custom dashboards to track workflow performance using built-in outputs and metrics. Thanks to a standardized format that Kestra uses, all your Kestra metrics can be visualized in Grafana or Kibana.
Stylized view of a monitoring dashboard

Simplicity scales, complexity fails

Save time and development costs.

Less time spent writing boilerplate orchestration code, more time for what matters to your business

Deploy and Run Anywhere

From on-premise installations to cloud-based services, containers to clusters, Kestra can be deployed anywhere.

Seamless Integrations

Connect to various data sources and cloud services like AWS, GCP, and Microsoft Azure.

Scale to Millions Executions

Kestra has been built with leading cloud-native technologies and a strong architecture, enabling scaling to millions of executions without hassle.

Logos of tools that integrate with Kestra such as Snowflake, Airbyte, DBT or Fivetran and Kestra at the center of It

Simple Integration

Customize and extend Kestra's integrations with custom plugins, tailored to your specifics needs. Integrate internal systems, third-party APIs, cloud services or databases using simple and scalable integrations.

See open-source pluginsCreate a custom plugin

Seamless Workflow Design and Management

Create your Flows, Avoid the Headache

Simplicity

Easily create, configure, and modify flows with Kestra's YAML format and user interface.

Helpful

Error spotting directly in the editor before any runtime error.

Allow other programming languages

Run any scripting language like Python, Bash, and Node.js using Kestra.

Use workflows as code

Deploy workflows like software code for easier collaboration and tracking of changes.

File processing capabilities

Files processing including CSV and JSON.

Versioning

Version workflows, tasks and data, rollback and execute a previous version track changes over time.

Reusability

Define generic tasks and workflows that can be used in different contexts.

Data validation

Ensures incoming and outgoing data in the pipeline steps comply with schema and quality constraints.

UI testing

Execution testing directly in the interface for fast Q&A before deployment.

Debugging

Advanced debugging tools directly in the flow editing interface for faster identification and troubleshooting.

No more deployment complexity

No need to create a dev environment to package and deploy an application, just work on the browser and use CI/CD.

Accelerate Development and Deployment

Build Faster, Ship Better

CI/CD Deployment

Integrate Kestra into CI/CD pipelines to automate the deployment of workflows (Terraform, GitLab Actions, GitLab, Jenkins…).

Production made easy, no more production plan

Kestra supports flow dependencies to define the execution order based on their interdependence.

Real-Time Events

Flow execution based on events from the core or plugins, such as the result of other flows in real-time, detection of files from SFTP or Google Cloud Storage, or results of an SQL query.

Event-driven Triggers

Endless possibilities to start a flow through API, using a webhook, waiting for a trigger, or simply starting directly from the Web UI.

Monitoring

Dashboards and metrics directly in the User Interface.

Error Handling

Handle errors automatically and retry failed tasks.

Notifications

Alerting configuration directly in the UI to alert when workflow tasks are completed, have errors or delays, or for any other important event.

Large Data Processing

Massive data volume handling with high performance using partitioning and batch processing techniques.

Micro-Batch for Real-Time

All the benefits of a real-time system without maintenance and complexity costs by collecting data at regular intervals and processing them in micro-batch.

No single point of failure

Horizontal and vertical scaling without worrying about resource allocation.

Performance Gains

Better performance with JVM (Java Virtual Machine) technology.

Centralized Data Control for Improved Efficiency

Data at Heart with Internal Storage

Abstraction of Files

The internal Storage feature abstracts files, enabling each task to produce or consume files without needing to know the server's current location.

Stateless Servers

Allows easy destruction, restart from scratch or restart a previous execution without any hassle.

Metadata Storage

Metadata from the flow is stored on a database and Kestra's Internal Storage for transparent file systems.

Strongly Typed

Ensure each piece of data is the right type and well-defined, a date must be a date, an int must be an int, and so on…

Amazon Ion Integration

Handles more types than JSON, to enable easy mixing of tasks usage without any pain.

Communication Between Tasks

Each task can easily communicate with all others, making it simple to mix tasks usage without any problems.

Task Switch flexibility

All switch between tasks, such as BigQuery to MySQL or BigQuery to Redshift, without needing specific operators.

Security and high availability

Take Kestra to the next level with the Enterprise Edition 🚀

Ensure scalability and high availability with Kafka and ElasticSearch Features available on the Enterprise Edition to eliminate all points of failure. Kestra's architecture is designed to handle high volumes of data and to ensure that workflows are processed reliably by default.

Secure your Kestra cluster with a single sign-on using your Open ID provider and control user and group access with fine-grained ACL.

Configure a namespace with task defaults, variables, and secrets to automatically configure all flows.
A design of the features of Kestra's enterprise edition with SSO connection, Role Based Access Control, Kafka and Elasticsearch integration

Service account

Create service accounts allowing to trigger workflows for external systems.

Secrets

Integrate your secret manager to ensure the security of your passwords and keys.

Audit Log

Audit all the resources created in one instance.