Fan Out a Sifflet Rule Pack with ForEach

A data domain is rarely guarded by one rule. Freshness, volume, and schema checks pile up, and running them as a coherent pack usually means a script nobody maintains or a person clicking through the Sifflet UI. This blueprint makes the pack a flow: io.kestra.plugin.core.flow.ForEach iterates over a JSON list of rule IDs, io.kestra.plugin.sifflet.RunRule launches each run with the iteration value, and Discord gets one summary message with the count instead of a message per rule.

How it works

1. The rule_ids input holds the pack as a JSON array. Editing the list is the whole maintenance story, no code changes when a rule is added or retired.
2. run_each_rule (io.kestra.plugin.core.flow.ForEach) renders values from the input and creates one child task run per rule ID.
3. Inside the loop, run_rule (io.kestra.plugin.sifflet.RunRule) reads the current ID from {{ taskrun.value }} and launches the rule run against your tenant.
4. If any launch fails, the execution fails and the errors block posts a Discord alert; the execution view shows exactly which iteration broke.
5. On success, notify_summary posts one message with the pack size computed from {{ inputs.rule_ids | length }}.
6. A disabled-by-default Schedule trigger runs the pack nightly at 05:00, after batch loads settle.

What you get

• One execution per pack run, with a per-rule task run inside it, instead of scattered individual runs.
• A rule pack defined as data, a JSON list anyone can edit without touching flow logic.
• A single summary notification with the count, and a distinct alert that fires on any failed launch.
• Per-iteration visibility in the execution view, so the broken rule ID is one click away.

Who it's for

• Data platform teams who group Sifflet rules by domain and want each group to run and report as a unit.
• On-call engineers who want one Discord message per pack, not one per rule.
• Anyone maintaining a shell script that curls the Sifflet API in a loop and would rather delete it.

Why orchestrate this with Kestra

Looping over an API is easy; making the loop observable is not. Kestra gives every iteration its own task run with logs and state, fails the pack loudly when one launch breaks, and keeps the rule list as a declared input rather than a hardcoded array in a script. The same pattern scales from two rules to two hundred by editing one JSON list.

Prerequisites

• The Sifflet rule IDs you want to group into a pack.
• A Sifflet API key with permission to run rules.
• A Discord incoming webhook for the summary and failure alerts.

Secrets

• SIFFLET_API_KEY: Sifflet API bearer token used to run the rules.
• SIFFLET_BASE_URL: base URL of your Sifflet tenant API.
• DISCORD_WEBHOOK_URL: Discord incoming webhook URL.

Quick start

1. Add the SIFFLET_API_KEY, SIFFLET_BASE_URL, and DISCORD_WEBHOOK_URL secrets to your Kestra namespace.
2. Set the rule_ids input to two or three real rule IDs and execute the flow.
3. Check Discord for the count summary, then grow the list to the full pack and enable the nightly trigger.

How to extend

• Set concurrencyLimit on the ForEach to launch several rule runs in parallel when the pack is large.
• Chain the pack behind your ingestion, as shown in the Sifflet rule after load blueprint, so checks always follow fresh data.
• Put a publish step behind the pack, as shown in the Sifflet pre-dashboard gate blueprint, to gate on the whole domain at once.
• Keep one pack per data domain by cloning the flow and changing only the rule_ids default.

Links

• Sifflet plugin documentation
• Sifflet documentation