WorkingDirectory icon
Clone icon
DbtCLI icon
Docker icon

Execute dbt transformations on Snowflake using a Git-based workflow

Run dbt ELT pipelines on Snowflake from Git using Docker and Kestra. Execute dbt deps and dbt build to transform data in your Snowflake warehouse.

Categories
Data

Run your dbt models directly on Snowflake by pulling the dbt project from Git and executing it inside a clean, reproducible Docker container. This blueprint solves the dependency-drift and "works on my machine" problem that plagues dbt teams: every run starts from version-controlled SQL and a pinned dbt image, so transformations behave identically across local development, CI, and production. It pairs dbt Core, Snowflake, Git, and containerized execution into a single declarative workflow, with warehouse credentials injected at runtime as secrets rather than committed to a profiles file.

How it works

  1. A io.kestra.plugin.core.flow.WorkingDirectory task creates a shared, isolated working directory so the cloned project and the dbt run share the same filesystem.
  2. Inside it, io.kestra.plugin.git.Clone checks out the dbt project from https://github.com/kestra-io/dbt-example on the main branch.
  3. io.kestra.plugin.dbt.cli.DbtCLI runs the project using the io.kestra.plugin.scripts.runner.docker.Docker task runner with the ghcr.io/kestra-io/dbt-snowflake:latest image. The Snowflake connection is defined in an inline profiles block (account, user, password, role, database, plus warehouse: COMPUTE_WH, schema: public, and threads: 4).
  4. The flow executes dbt deps to install packages, then dbt build to run and test all models in dependency order.

What you get

  • Reproducible dbt runs from a pinned Docker image, no local dbt install required.
  • Analytics logic that stays versioned and reviewable in Git.
  • Snowflake credentials kept out of code and injected as secrets at runtime.
  • Models and tests executed together through dbt build for trustworthy data.

Who it's for

  • Analytics engineers building and testing dbt models on Snowflake.
  • Data platform teams standardizing dbt execution across environments.
  • Teams moving dbt off laptops and ad hoc cron into managed orchestration.

Why orchestrate this with Kestra

dbt's own CLI and scheduler can run models, but they cannot react to upstream events, retry failed steps, or stitch dbt into a wider pipeline. With Kestra you define the whole run in declarative YAML, add event or schedule triggers, attach retries and alerting, and capture execution lineage and logs in one place. The same flow can sit downstream of ingestion and upstream of reverse ETL or reporting, something a standalone dbt invocation cannot coordinate.

Prerequisites

  • A running Kestra instance with Docker available to the task runner.
  • A Snowflake account with a warehouse, database, and role for dbt.
  • A dbt project in Git (this blueprint uses the public kestra-io/dbt-example).

Secrets

  • SNOWFLAKE_ACCOUNT
  • SNOWFLAKE_USER
  • SNOWFLAKE_PASSWORD
  • SNOWFLAKE_ROLE
  • SNOWFLAKE_DATABASE

Quick start

  1. Add the five Snowflake secrets to your Kestra instance.
  2. (Optional) Point the io.kestra.plugin.git.Clone url and branch at your own dbt repository.
  3. Adjust the profiles block to match your warehouse, schema, and target.
  4. Save the flow and trigger an execution, then inspect the dbt logs in the run output.

How to extend

  • Replace dbt build with dbt run plus dbt test, or add dbt docs generate.
  • Add a io.kestra.plugin.core.trigger.Schedule or event trigger to run on a cadence or after ingestion.
  • Parameterize the Git branch, target, or warehouse with inputs for multi-environment runs.
  • Chain downstream tasks such as Slack alerts or reverse ETL after a successful build.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.