Blueprints

Git workflow for dbt with Google BigQuery

Source

yaml
id: dbt-bigquery
namespace: company.team

tasks:
  - id: git
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: main

      - id: dbt
        type: io.kestra.plugin.dbt.cli.DbtCLI
        inputFiles:
          sa.json: "{{ secret('GCP_CREDS') }}"
        taskRunner:
          type: io.kestra.plugin.scripts.runner.docker.Docker
        containerImage: ghcr.io/kestra-io/dbt-bigquery:latest
        profiles: |
          my_dbt_project:
            outputs:
              dev:
                type: bigquery
                dataset: your_big_query_dataset_name
                project: your_big_query_project
                keyfile: sa.json
                location: EU
                method: service-account
                priority: interactive
                threads: 16
                timeout_seconds: 300
                fixed_retries: 1
            target: dev
        commands:
          - dbt deps
          - dbt build
See more

About this blueprint

BigQuery Git dbt

This flow:

  • clones a dbt Git repository from GitHub,
  • pulls a public container image with the required dependencies
  • runs dbt CLI commands within a container.

This flow assumes that your Google Cloud Service Account JSON key file's content is stored as a secret named GCP_CREDS.

Before you run this blueprint, simply copy-paste the contents of your ~/.dbt/profiles.yml into the profiles section and you're good to go!

Working Directory

Clone

Dbt CLI

Docker

More Related Blueprints

Parallel Ingest BigQuery Git dbt

Trigger multiple Airbyte Cloud syncs in parallel, then run a dbt job

Parallel Ingest BigQuery Git dbt

Trigger multiple Airbyte syncs, then run a dbt job

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra