Data Quality Gate on Trino with Thresholds and Deliberate Failure

A quality check that only writes a warning to a log is a quality check everyone ignores. This blueprint makes the check binding: one io.kestra.plugin.jdbc.trino.Query with fetchType: FETCH_ONE measures the null rate and row count of yesterday's partition, an If task compares both numbers against thresholds, and on breach the flow posts a Slack alert and then fails the execution with io.kestra.plugin.core.execution.Fail. A failed execution is a signal other flows can depend on, so nothing downstream builds on a bad partition.

How it works

1. check_partition (io.kestra.plugin.jdbc.trino.Query, fetchType: FETCH_ONE) computes row_count and null_rate for iceberg.analytics.events where event_date is yesterday. The single result row is addressable as {{ outputs.check_partition.row.* }}.
2. quality_gate (io.kestra.plugin.core.flow.If) evaluates null_rate > 0.05 or row_count < 1000.
3. On breach, alert_breach posts the exact measured values and both thresholds to Slack, then fail_execution (io.kestra.plugin.core.execution.Fail) fails the run with an errorMessage carrying the same numbers.
4. When the partition is healthy, log_pass writes the metrics to the execution log so every day has an auditable record.
5. The errors block posts a Slack alert for any failure, including the deliberate one, and a disabled-by-default Schedule trigger runs the gate daily at 06:00.

What you get

• A binding quality contract: breaches fail the execution instead of scrolling past in a log.
• Exact measured values in both the Slack alert and the failure message, so triage starts with numbers.
• A green-or-red execution history that doubles as a quality audit trail per day.
• Thresholds expressed in one readable Pebble condition, reviewable in version control.

Who it's for

• Data engineers who need a circuit breaker between ingestion and consumption.
• Analytics engineers tired of dashboards quietly rendering from half-loaded partitions.
• Platform teams standardizing lightweight quality gates without adopting a separate framework.

Why orchestrate this with Kestra

The SQL is trivial; the enforcement is not. Kestra provides the pieces around the query that make the gate real: a schedule, a conditional branch, a first-class way to fail an execution with a meaningful message, notifications, and an execution history where every breach is visible and replayable. Downstream flows can key off this flow's success state, which turns one small check into a dependency contract.

Prerequisites

• A Trino cluster with a table matching iceberg.analytics.events partitioned or filterable by an event_date column, or adjust the SQL.
• Thresholds that fit your data; 0.05 null rate and 1000 rows are starting points, not universal truths.
• A Slack incoming webhook for alerts.

Secrets

• TRINO_URL: JDBC URL, e.g. jdbc:trino://host:443/iceberg/analytics (use https on 443 in production; password authentication requires TLS).
• TRINO_USERNAME: Trino username.
• TRINO_PASSWORD: Trino password.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the four secrets to your Kestra namespace.
2. Point the SQL at your table, date column, and the column whose null rate matters.
3. Execute the flow and verify the pass path logs metrics; temporarily raise the row threshold to see the breach path fire.
4. Set disabled: false on the daily trigger.

How to extend

• Add more metrics to the same one-row query (duplicate rate, min and max timestamps, distinct counts) and extend the condition.
• Make thresholds flow inputs so operators can tune them without editing the flow.
• Chain the downstream load as a subflow that only runs after this flow succeeds, or use a Flow trigger conditioned on this flow's SUCCESS state.
• Run the gate per table with io.kestra.plugin.core.flow.Loop over a table list.

Links

• Trino plugin documentation
• Trino documentation