Airbyte

Airbyte is the open source data integration platform with hundreds of connectors. Kestra decides when each sync runs, waits for it to land, and runs everything that comes after it.

  • Technology partner
  • United States
  • North America

About

The plugin-airbyte plugin triggers syncs on Airbyte OSS through connections.Sync and on Airbyte Cloud through cloud.jobs.Sync, and turns each connector run into a step of a larger declarative flow.

“Kestra has been a keystone to design complex execution flows while enhancing our Infrastructure as Code best practices. It now empowers our ingestion with Airbyte, transformation with dbt, and our Hightouch reverse-ETL jobs seamlessly.”
Antoine Balliet, Senior Data Engineer at Gorgias

Airbyte + Kestra: ingestion, orchestrated end to end

  • Syncs that start when the data is ready

    A webhook, a file landing in S3 or a schedule fires io.kestra.plugin.airbyte.connections.Sync, so ingestion runs on a real signal instead of a fixed cron.

  • Wait for the job to finish

    Sync waits for a terminal status by default, and CheckStatus polls a job started elsewhere. The next task only reads fully loaded tables.

  • Ingestion, dbt and activation in one flow

    Run the Airbyte sync, build dbt models, then trigger Hightouch. A failed dbt step retries on its own without running the sync again.

  • Many connections in parallel

    Fan out syncs with Loop and a concurrency limit, and isolate a broken connector with allowFailure so the rest of the refresh completes.