WorkingDirectory icon
Clone icon
DbtCLI icon
Docker icon

Run dbt models on Amazon Redshift directly from a Git repository

Run dbt ELT pipelines on Amazon Redshift from Git using Docker and Kestra. Execute dbt deps and dbt build to transform data in your Redshift warehouse.

Categories
Data

Run your dbt transformations on an Amazon Redshift data warehouse straight from a version-controlled Git repository, with zero local setup. This blueprint clones a dbt project, resolves its dependencies, and executes dbt build inside a pinned Docker image, so the same dbt models, tests, and snapshots run identically across development, staging, and production. It solves the classic analytics engineering problem of "it works on my machine" by making every dbt run reproducible, isolated, and driven entirely from source control.

How it works

  1. A io.kestra.plugin.core.flow.WorkingDirectory task named git creates a shared working directory so the cloned repo and the dbt run happen in the same context.
  2. Inside it, clone_repository (io.kestra.plugin.git.Clone) checks out the main branch of the dbt project from a Git URL.
  3. The dbt task (io.kestra.plugin.dbt.cli.DbtCLI) runs on a Docker task runner (io.kestra.plugin.scripts.runner.docker.Docker) using the ghcr.io/kestra-io/dbt-redshift:latest image, which already bundles dbt and the Redshift adapter.
  4. An inline dbt profiles block defines a redshift target, pointing at your cluster host, port 5439, database, and schema. Credentials are injected at runtime via {{ secret('REDSHIFT_USER') }} and {{ secret('REDSHIFT_PASSWORD') }}.
  5. The task executes dbt deps to install packages, then dbt build to run models, tests, snapshots, and seeds in dependency order.

What you get

  • Reproducible dbt runs from a pinned container image, no local Python or adapter installs.
  • Version-controlled transformations sourced directly from Git on every execution.
  • Secure credential handling: Redshift user and password stay out of the repository.
  • A single declarative YAML definition covering clone, dependency resolution, and build.

Who it's for

  • Analytics engineers running dbt models on Redshift.
  • Data teams standardizing dbt execution across environments.
  • Platform engineers who want dbt runs orchestrated alongside ingestion and downstream jobs.

Why orchestrate this with Kestra

dbt's own CLI runs models, but it does not schedule, trigger, retry, or connect dbt to the rest of your pipeline. With Kestra you can fire this build from event triggers (a new file landing, an upstream load finishing, a webhook, or a cron schedule), add automatic retries on transient Redshift or network failures, capture full run logs and lineage, and chain dbt before or after ingestion and reverse-ETL steps. Everything stays in declarative, reviewable YAML stored in Git, filling the gap dbt Core's scheduler leaves open.

Prerequisites

  • A running Amazon Redshift cluster reachable from Kestra, with a database and target schema.
  • A dbt project in a Git repository (this blueprint defaults to a public example).
  • Docker available to the Kestra worker for the task runner.

Secrets

  • REDSHIFT_USER: the Redshift database user for the dbt connection.
  • REDSHIFT_PASSWORD: the password for that Redshift user.

Quick start

  1. Add the REDSHIFT_USER and REDSHIFT_PASSWORD secrets to your Kestra instance.
  2. Update the url in clone_repository to point at your own dbt project repository.
  3. Edit the dbt profiles block: set host, dbname, and schema for your Redshift cluster.
  4. Run the flow and confirm dbt deps and dbt build complete against Redshift.

How to extend

  • Swap dbt build for dbt run, dbt test, or a targeted --select to control scope.
  • Add a io.kestra.plugin.core.trigger.Schedule trigger to run transformations on a cron cadence.
  • Pin the container image to a specific tag instead of latest for stricter reproducibility.
  • Parameterize the Git branch or target environment with inputs to support staging and production.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.