Run a Sifflet Rule After a Warehouse Load

Most data quality checks run on their own schedule and hope the load finished first. When ingestion runs late, the rule evaluates yesterday's tables, passes, and everyone trusts numbers that were never actually checked. This blueprint removes the race: the ingestion step and the Sifflet rule run live in one Kestra execution, so io.kestra.plugin.sifflet.RunRule only fires once the data has landed, and Slack gets the run status with the execution ID.

How it works

1. load_warehouse stands in for your ingestion. It is a plain Log task here so the flow runs anywhere; in production you replace it with a real load such as the dlt REST API to DuckDB blueprint or a Trino query chain, keeping the rule run chained behind it.
2. run_quality_rule (io.kestra.plugin.sifflet.RunRule) authenticates with the API key, targets your tenant through baseUrl, and calls Sifflet's rule run endpoint for the rule_id input.
3. The task exposes status, statusCode, ruleId, and the raw response as outputs; notify_team interpolates the status and HTTP code into a Slack message.
4. The errors block posts a distinct alert whether the load or the rule run failed, and the execution view shows which half broke.
5. A disabled-by-default Schedule trigger runs the whole chain at 06:00 so data is loaded and checked before standup.

What you get

• A quality check that runs when data lands, not on a blind timer that races the load.
• The Sifflet run status and rule ID as first-class flow outputs, ready for notifications or downstream conditions.
• One execution history entry that answers whether the data loaded, whether the rule ran, and who was told.
• A chain where swapping the ingestion half changes one task, not the architecture.

Who it's for

• Data engineers who own both the load and the observability rules and are tired of aligning their schedules by hand.
• Data platform teams rolling out Sifflet who want rule runs wired into pipelines instead of running beside them.
• Anyone who has debugged an incident where the quality check passed because it ran before the bad load arrived.

Why orchestrate this with Kestra

Sifflet can schedule its own rule runs, but it cannot see your warehouse loads. Kestra sequences the two with explicit dependencies: the rule run starts only after the load succeeds, the run status becomes a flow output that feeds notifications, and failures page Slack instead of leaving unchecked data in production. The unchanged Sifflet rule just gets called at the right moment.

Prerequisites

• A Sifflet rule and its rule ID, visible in the rule URL or through the Sifflet API.
• A Sifflet API key with permission to run rules.
• A Slack incoming webhook for run reports.

Secrets

• SIFFLET_API_KEY: Sifflet API bearer token used to run the rule.
• SIFFLET_BASE_URL: base URL of your Sifflet tenant API.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the SIFFLET_API_KEY, SIFFLET_BASE_URL, and SLACK_WEBHOOK_URL secrets to your Kestra namespace.
2. Set the rule_id input to a real Sifflet rule and execute the flow.
3. Check Slack for the run status, then replace load_warehouse with your real ingestion and enable the daily trigger.

How to extend

• Replace the stand-in load with the dlt REST API to DuckDB blueprint, a Trino query, or any warehouse task, and keep the rule run chained behind it.
• Fan out to a whole rule pack by wrapping the task in a ForEach over rule IDs, as shown in the Sifflet rule pack fan-out blueprint.
• Gate a publish step on the rule run instead of just reporting it, as shown in the Sifflet pre-dashboard gate blueprint.
• Raise requestTimeout above its 30 second default if your tenant sits behind a slow gateway.

Links

• Sifflet plugin documentation
• Sifflet documentation