Announcing Kestra Technology Partnership with Airbyte
We are pleased to announce a technology partnership with Airbyte. Kestra and Airbyte share a common vision for the future of the Modern Data Stack. This technical partnership ensures seamless data transfer, task execution, and process automation between Kestra and Airbyte.
“Airbyte and Kestra share a common vision: to simplify data pipeline creation. This partnership brings together the strengths of two leading data management companies, so our customers can get the most out of their data and achieve their business goals.”
Chris Tata Head of Partnerships @ Airbyte
With technology partners like Airbyte, we're able to cover more use cases in our existing framework. By integrating with Airbyte, Kestra can provide an open technology stack that can integrate, transform, process, and move data at any scale.
“Kestra excels at orchestrating data pipelines, and often the most challenging part of this is the initial data integration, a stage where Airbyte shines. The collaboration between Kestra and Airbyte, both open-source platforms, is a natural fit as we share a common mission: to simplify and optimize data solutions for business users.”
Ludovic Dehon CTO @ Kestra
Kestra's plugin for Airbyte enables users to trigger Airbyte syncs and optionally, wait for their completion. Whether you use the open-source Airbyte Server or Airbyte Cloud, Kestra can integrate your data ingestion process into an end-to-end data workflow.
Our collaboration highlights an innovative approach to data management—treating everything as code. The partnership strengthens the ability to automate data operations and infrastructure deployment, which means you can now manage, modify, and scale your data pipelines more efficiently.
- Everything-as-Code Approach: Manage data operations like code, fostering collaboration, and efficient scaling. More insights can be found in this detailed blog post on everything-as-code for data infrastructure.
- Terraform Providers Integration: Both platforms offer strong Terraform support, enabling you to manage and provision resources using declarative configuration files.
The collaborative efforts offer a unified platform for end-to-end data orchestration, allowing users to build data workflows with ease:
- Dynamic Data Workflows: You can use Kestra's Airbyte plugin to trigger various operations, such as parallel syncs, scheduled ingestion, and complex data processing jobs.
- Robust Orchestration Capabilities: From triggering multiple Airbyte Cloud syncs to running complex dbt jobs, Kestra's flexibility empowers users to control and automate their data processes.
- Efficient Data Management: Learn how this collaboration supports an end-to-end data orchestration approach in this blog post.
With the Kestra-Airbyte collaboration, users can perform a wide range of tasks:
- Trigger Multiple Airbyte Cloud Syncs in Parallel: Facilitating efficient data synchronization.
- Scheduled Syncing: Planning single or multiple Airbyte syncs.
- Complex Data Operations: Execute intricate tasks, such as triggering multiple Airbyte syncs and then running dbt jobs.
This partnership provides a versatile toolkit for data professionals, extending the possibilities of data handling. The potential applications are large, aligning with various organizational needs and ensuring data-driven success. Discover more about this integration through our documentation.
If you want to go further with Kestra and Airbyte capabilities you can read this article about Everything as Code for Data Infrastructure with Airbyte and Kestra Terraform Providers or how to create an End-to-End Data Ingestion, Transformation and Orchestration pipeline with Airbyte, dbt and Kestra
You can also kickstart you Airbyte & Kestra Journey with our Community Blueprints