Run dbt ELT pipelines on Postgres from Git using Docker

Run a production-ready dbt ELT pipeline on PostgreSQL by pulling a dbt project straight from a Git repository and executing it inside a reproducible Docker container. This blueprint solves a common analytics engineering problem: keeping dbt transformations version-controlled in Git while running them on a schedule or on demand against operational and analytical Postgres databases, without manually managing the dbt environment, dependencies, or database credentials on a host machine.

How it works

1. A io.kestra.plugin.core.flow.WorkingDirectory task (git) creates an isolated working directory so the cloned project and dbt commands share the same filesystem context.
2. Inside it, io.kestra.plugin.git.Clone (clone_repository) clones the dbt project from the main branch of the configured GitHub url.
3. io.kestra.plugin.dbt.cli.DbtCLI (dbt) runs in the ghcr.io/kestra-io/dbt-postgres:latest container image with a postgres profile. Credentials are injected through {{ secret('POSTGRES_USER') }} and {{ secret('POSTGRES_PASSWORD') }}, never hardcoded.
4. The task executes dbt deps to install package dependencies, then dbt build to run models, tests, snapshots, and seeds against the dev target.

What you get

• A version-controlled dbt project executed in a clean, reproducible Docker environment on every run
• Secure Postgres connectivity using Kestra secrets instead of plaintext credentials
• Models, tests, and snapshots built in a single dbt build step
• The same workflow reusable across development, staging, and production

Who it's for

• Analytics engineers running dbt on PostgreSQL (Amazon RDS, Cloud SQL, or self-managed)
• Data teams standardizing dbt execution with Docker and Git
• Platform engineers who want scheduled, observable dbt runs without bespoke scripts

Why orchestrate this with Kestra

dbt itself has no scheduler, retry logic, or event triggers. Kestra adds event-driven and scheduled triggers, automatic retries on transient database or network failures, full execution logs and lineage across the clone and build steps, and a declarative YAML definition you can keep in Git alongside the dbt project. You also get a clear handoff point to chain dbt runs with upstream ingestion or downstream notifications, which dbt's own tooling cannot coordinate.

Prerequisites

• A reachable PostgreSQL database (update the host, port, dbname, and schema in the profile to match yours)
• A dbt project in a Git repository (the example points at kestra-io/dbt-example)
• Docker available to the Kestra worker to pull the dbt-postgres image

Secrets

• POSTGRES_USER: the PostgreSQL user dbt connects as
• POSTGRES_PASSWORD: the password for that user

Quick start

1. Add the POSTGRES_USER and POSTGRES_PASSWORD secrets to your Kestra instance.
2. Point the clone_repository url and branch at your own dbt repository.
3. Edit the profiles block so host, port, dbname, and schema match your database.
4. Run the flow and review the dbt logs in the execution outputs.

How to extend

• Add a io.kestra.plugin.core.trigger.Schedule trigger to run the pipeline on a cron cadence.
• Swap the dev target for prod or parameterize the profile with flow inputs per environment.
• Replace dbt build with dbt run plus dbt test, or add dbt docs generate to publish documentation.
• Chain an upstream ingestion task and a downstream Slack or email notification around the dbt step.

Links

• Plugin documentation
• dbt Core orchestration with Kestra