Execute dbt Models on BigQuery with Automated Slack Incident Alerting

Data teams relying on dbt for Google BigQuery modeling require dependable daily transformation pipelines that execute cleanly, catch failing data contracts early, and alert on-call data engineers immediately when model compilation or schema tests fail.

This blueprint orchestrates containerized dbt build operations targeting Google BigQuery. It executes transformations inside an official dbt container, inspects exit codes and test outcomes with a dedicated Python validation task, and conditionally dispatches rich notification payloads to your designated Slack operations channel.

How it works

1. run_dbt_models (io.kestra.plugin.scripts.shell.Commands): Spawns the official ghcr.io/dbt-labs/dbt-bigquery container via Docker task runner and executes the dbt compilation and run cycle against BigQuery.
2. inspect_execution_results (io.kestra.plugin.scripts.python.Script): Analyzes the generated test manifest and run metrics, outputting structured status parameters via Kestra.outputs.
3. notify_conditional_status (io.kestra.plugin.core.flow.If): Evaluates whether both the CLI exit code and schema validations passed. If successful, dispatches notify_slack_success; otherwise, routes to notify_slack_failure.
4. alert_on_pipeline_failure (errors block): Intercepts fatal runner aborts or network dropouts to immediately notify engineers via Slack with the execution ID.

What you get

• Containerized, isolated dbt runtime running directly against Google BigQuery without local environment drift.
• Automated daily scheduling alongside on-demand manual triggers.
• Dual notification paths distinguishing between clean builds, data test warnings, and fatal runner crashes.
• Full auditability with linked Kestra execution tracking.

Who it's for

• Analytics Engineers operating dbt models at scale on Google Cloud BigQuery.
• Data Platform Engineers building automated ELT orchestrations with alerting.
• BI Teams requiring verified data freshness before morning dashboard refreshes.

Why orchestrate this with Kestra

Kestra simplifies dbt operations by decoupling transformation orchestration from underlying infrastructure. Through Docker runner isolation, granular secrets management for Google Cloud Service Accounts, and event-driven alerting via Slack, Kestra provides a robust, observable data platform control plane without needing dedicated Kubernetes orchestration clusters.

Prerequisites

• Google Cloud Project with BigQuery enabled and a Service Account with BigQuery Data Editor and Job User roles.
• Slack Incoming Webhook URL configured for incident dispatch.

Secrets

• GCP_CREDS: Google Cloud Service Account JSON credentials.
• SLACK_WEBHOOK_URL: Slack Incoming Webhook endpoint for alerting.

Quick start

1. Configure the GCP_CREDS and SLACK_WEBHOOK_URL secrets in your Kestra namespace.
2. Import this blueprint into your Kestra instance.
3. Customize the dbt_target and slack_channel inputs to fit your organizational setup.
4. Click Execute to trigger a test run.
5. Enable the daily_dbt_schedule cron trigger for automated daily execution.

How to extend

• Add upstream extraction triggers (e.g. Fivetran, Airbyte, or GCS file arrivals) using Kestra webhook or polling triggers.
• Connect downstream BI cache warming tasks (e.g. Looker or Tableau refresh APIs) after the successful dbt build.
• Add data diffing checks using Datafold or elementary-data plugins for regression detection.

Links

• Kestra dbt Plugin Documentation
• Kestra Slack Plugin Documentation
• dbt-bigquery Documentation
• This blueprint has been created by GauravPatil2515