Trigger dbt build on cron, after an Airbyte sync, or when a file lands in S3. Capture per-model test failures, retry only the failed step, replay any model from history, and chain Airbyte upstream and Hightouch downstream in one execution.
Run dbt Core with a workflow engine that listens to cron, Airbyte completions, or S3 events, handles per-model retries with approval gates between staging and prod, and chains dbt with Airbyte, Hightouch, and the rest of your stack in one audited flow. West Bend Insurance estimates ~90% less Python glue code with an orchestrator around dbt.
dbt Core solves transformation. Kestra solves everything around it: when it runs, what triggers it, what happens next, and who ran what and when.
dbt Core is a single step. It has no scheduler, no event triggers, and no awareness of the pipeline around it. Kestra treats dbt as one node in a flow: it retries only the failed step, passes outputs forward, and halts downstream tools until the data is confirmed ready.
A model output changes. A test starts failing. A downstream dashboard shifts. With cron or GitHub Actions there is no easy answer to who ran what, against which data state. Kestra logs every execution with a trigger source, a timestamp, and the full test result. The record exists before you need it.
When dbt test fails in CI, finding out which model failed and why means digging through log output. Kestra captures every model's output, test result, and failure in a single execution view: timestamped, attributed, and searchable without log archaeology.
Patterns data engineering teams run in production today. Each one shows the flow end to end, with the real plugin classes in play.
Run dbt run and dbt test on a cron schedule. If any model or test fails, Kestra captures the failure context per model and routes it to Slack with the exact test name and assertion that failed. No one manually digs through CI logs to understand what broke.
Native scheduler with cron, fixed intervals, or condition-based triggers.
Slack / email routing with the failing test name and assertion attached.
Click straight to the failing model's execution view, no log archaeology.
Run dbt against a staging target first. Pass the output to a named reviewer. Only promote to the production target after explicit approval. Full audit trail of who approved what and when. Stops the pipeline before the production run if staging tests fail.
Built-in Pause task suspends execution until a reviewer approves.
Identity and approval moment stored in execution history automatically.
Pipeline halts before the production run if any staging test fails.
Every promotion linked to a reviewer and a moment in time, replayable.
Ingest raw data with Airbyte, transform it with dbt, then sync results downstream with Hightouch. All three run as steps in a single Kestra flow. If dbt fails, Kestra retries only the dbt step without re-running the Airbyte sync. This is the exact pattern Gorgias runs in production.
Retry only the failed step, no re-running of the ingestion.
Pass artifacts and metadata between Airbyte, dbt, and Hightouch.
One execution ID across all three tools. One place to debug.
Search every run by date, status, or trigger source from one UI.
Trigger dbt when a new file lands in S3, when an Airbyte sync completes, or when an API webhook fires. No cron job, no polling script. Kestra fires the dbt pipeline the moment the data is ready, using native event triggers.
File arrivals and message events as first-class triggers.
Fire dbt from any HTTP call, no glue service required.
React to row inserts, updates, or schema changes.
Workflows start the moment data is ready, not on the next interval.
Run only the dbt models that need refreshing (by tag, path, or selector) rather than a full project rebuild. Pass the model selector as a flow input, so the same flow handles both nightly full runs and on-demand targeted refreshes triggered by specific upstream changes.
Pass model selectors and target names as flow parameters at run time.
Nightly full builds and on-demand targeted refreshes share one YAML.
Use any dbt selector syntax. Kestra forwards it unchanged.
Kestra has been a keystone to design complex execution flows while enhancing our Infrastructure as Code best practices. It now empowers our ingestion with Airbyte, transformation with dbt, and our Hightouch reverse-ETL jobs seamlessly!
| Capability | | dbt Cloud | GitHub Actions | |
|---|---|---|---|---|
| Scheduling | Native cron + events | Native job scheduling | Cron-based workflows | Native DAG scheduler |
| Approval gate before production run | Native pause + resume | No pause-and-resume | Environment approval | Custom sensor workaround |
| Automatic retry per step | Configurable per task | Job-level retry only | Job re-run from start | Task-level retry |
| Chain with upstream tools (Airbyte, Spark) | Native, with outputs | dbt only | Possible, complex | Possible, operator-dependent |
| Per-model failure visibility | Execution history + task state | In dbt Cloud UI | Logs in Actions UI | Task logs, no dbt detail |
| Self-service for non-engineers | Kestra Apps | Admin-controlled jobs | No | No |
| Orchestrate beyond dbt (full stack) | 1300+ plugins | dbt only | Possible, fragile | Yes, with Python operators |
Find answers to your questions right here, and don't hesitate to Contact Us if you couldn't find what you're looking for.
Schedule, chain, and visualize every dbt run across your full data stack. Open source, self-hosted, event-driven.
