DbtCLI icon
Sync icon
SlackIncomingWebhook icon
Docker icon
PagerDutyAlert icon

dbt Test Gate Before Hightouch Activation with PagerDuty Alert

Orchestrate a dbt test gate before a Hightouch reverse-ETL sync with Kestra. Activate only on passing tests, page PagerDuty on data quality failures.

Categories
Data

Reverse ETL is only as trustworthy as the data behind it, and the classic anti-pattern is pushing unvalidated rows straight into customer-facing systems like Salesforce or HubSpot. This blueprint puts a dbt test gate in front of every Hightouch activation: it builds the dbt models, runs dbt test as a separate task, and lets the sync fire only when every test passes. When tests fail, Kestra halts the flow and pages the on-call before a single bad record reaches a destination.

How it works

  1. The dbt_run task (io.kestra.plugin.dbt.cli.DbtCLI) runs dbt deps and dbt run to build the models that feed the activation sync.
  2. The dbt_test task (a separate io.kestra.plugin.dbt.cli.DbtCLI task) runs dbt test. Because it is its own task, a successful build with failing tests still stops the flow here.
  3. The hightouch_sync task (io.kestra.plugin.hightouch.Sync) activates the validated rows. It reads the hightouch_sync_id input, authenticates with the HIGHTOUCH_API_TOKEN secret, and uses wait: true to block until the run completes.
  4. The notify_success task (io.kestra.plugin.slack.notifications.SlackIncomingWebhook) posts a Slack confirmation with the Hightouch runId and the execution ID.
  5. If any task fails, the page_on_quality_failure error handler (io.kestra.plugin.pagerduty.PagerDutyAlert) triggers a PagerDuty Events API v2 alert.

What you get

  • A hard quality gate: no Hightouch sync runs unless dbt test passes.
  • Automatic paging to the data on-call when tests fail, before any destination is touched.
  • Slack confirmation with the Hightouch run ID for an auditable activation trail.
  • dbt running in a pinned Docker image (ghcr.io/kestra-io/dbt-bigquery:latest) with namespace files enabled.

Who it's for

  • Analytics engineers who own dbt marts that feed activation.
  • Data platform and RevOps teams running reverse ETL into CRMs.
  • On-call data engineers who need a fail-loud signal when quality drops.

Why orchestrate this with Kestra

Hightouch can schedule a sync, but it cannot run your dbt build, gate on dbt test, and page PagerDuty in one declarative pipeline. Kestra models the whole chain as YAML with explicit task ordering, so the activation depends on test success rather than a timer. You get retries, full execution lineage across dbt, Hightouch, Slack, and PagerDuty, and an errors handler that fires only on failure. Event triggers, webhooks, or a schedule can all kick off the same flow.

Prerequisites

A dbt project uploaded as namespace files, with tests defined on the activation marts. Swap the dbt-bigquery image for dbt-snowflake, dbt-postgres, or any other adapter as needed.

Secrets

  • HIGHTOUCH_API_TOKEN: API bearer token for Hightouch.
  • SLACK_WEBHOOK_URL: Slack incoming webhook URL for the success notification.
  • PAGERDUTY_ROUTING_KEY: PagerDuty Events API v2 routing key for the data-quality page.

Quick start

  1. Set the three secrets above in your Kestra namespace.
  2. Upload your dbt project as namespace files, with tests on the activation marts.
  3. Run the flow, passing the hightouch_sync_id input. PagerDuty pages only if dbt test fails.

How to extend

  • Add a schedule or event trigger so activation runs after each upstream load.
  • Tighten the PagerDuty routing key to the data on-call rotation.
  • Branch the Slack message to a failure channel, or add severity routing.
  • Activate multiple Hightouch syncs in parallel once tests pass.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.