Run dbt transformations on DuckDB from Git for local analytics and ELT

Run dbt Core transformations on DuckDB directly from a Git repository for fast, reproducible, and fully version-controlled analytics ELT. This blueprint clones a dbt project, executes dbt deps and dbt build inside a lightweight DuckDB container, and materializes models in an in-memory DuckDB database. It solves a common pain point: standing up dbt against a cloud warehouse just to test models, run CI, or prototype, when all you need is Git and Docker.

How it works

1. A io.kestra.plugin.core.flow.WorkingDirectory task named dbt creates a shared workspace so the clone and the dbt run operate on the same files.
2. The clone_repository task (io.kestra.plugin.git.Clone) pulls the main branch of the example dbt project from GitHub into that working directory.
3. The dbt_build task (io.kestra.plugin.dbt.cli.DbtCLI) runs on a Docker task runner (io.kestra.plugin.scripts.runner.docker.Docker) using the ghcr.io/kestra-io/dbt-duckdb:latest image.
4. It runs dbt deps to resolve packages, then dbt build to compile, run, and test models against DuckDB.
5. An inline profiles block points dbt at a duckdb target with path: ":memory:", so models materialize in-memory with no external database.

What you get

• A complete dbt build (models, tests, snapshots, seeds) on every run.
• Reproducible, containerized execution that needs no warehouse provisioning.
• dbt manifest and run artifacts captured as outputs in Kestra.
• A pipeline that runs identically on a laptop and in production.

Who it's for

• Analytics engineers prototyping or testing dbt models.
• Data teams running dbt CI without a cloud warehouse.
• Engineers evaluating dbt or DuckDB before a larger migration.

Why orchestrate this with Kestra

dbt's own scheduler is limited to time-based runs and cannot react to upstream events. With Kestra you add event triggers (a new commit, an arriving file, an API call), automatic retries, run-level lineage and logs, and a declarative YAML definition that lives in Git alongside your dbt project. The Docker task runner pins the exact runtime, so the same flow is portable across environments.

Prerequisites

• A Kestra instance with Docker available to the worker.
• Network access to the Git repository and the container registry.

Secrets

This blueprint uses no secrets. The example repository is public and DuckDB runs in-memory. Add secrets only when you point it at a private repo or a real warehouse, for example a Git token via {{ secret('GITHUB_TOKEN') }}.

Quick start

1. Add this flow to a namespace in your Kestra instance.
2. Confirm the worker can run Docker containers.
3. Trigger an execution and watch the dbt logs stream in the UI.
4. Inspect the outputs of dbt_build for the manifest and run results.

How to extend

• Swap the url in clone_repository for your own dbt project.
• Change the duckdb path to a persisted file to keep results between runs.
• Replace the DuckDB profile with Snowflake, BigQuery, or Postgres for production targets.
• Add a Git push or webhook trigger to run dbt on every merge.

Links

• dbt plugin documentation
• Orchestrate dbt Core with Kestra
• Git plugin documentation