Recheck a Sifflet Rule After an Incident Fix

Data incidents have a bad habit of being closed on optimism: the fix is deployed, the pipeline reran, and someone marks the ticket resolved without rerunning the check that caught the problem. This blueprint wires the recheck into the incident workflow. When your incident tooling reports a fix deployed, it calls a Kestra webhook, io.kestra.plugin.sifflet.RunRule launches a fresh run of the exact rule that fired, and Slack gets the verdict, so closing the incident becomes a decision backed by a rule run instead of a guess.

How it works

1. The on_fix_deployed trigger (io.kestra.plugin.core.trigger.Webhook) exposes an HTTP endpoint guarded by a secret key. PagerDuty, Opsgenie, a runbook button, or any tool that can POST fires the flow when a fix lands.
2. recheck_rule (io.kestra.plugin.sifflet.RunRule) authenticates with the API key, targets your tenant through baseUrl, and launches a run of the rule named by the rule_id input, the same monitor that detected the incident.
3. The task exposes status, statusCode, ruleId, and the raw response; post_verdict posts the status to the incident channel with an explicit instruction to review the run in Sifflet before closing.
4. The errors block posts a distinct alert when the recheck fails to launch, telling the resolver not to close the incident on it.
5. The execution history keeps a permanent record of every recheck, useful for the postmortem timeline.

What you get

• A recheck that runs the moment a fix is deployed, triggered by the incident tool instead of a human remembering.
• A Slack verdict in the incident channel with the rule ID, run status, and execution link material.
• Loud failure when the recheck cannot run, so incidents never close on a check that silently did not happen.
• A postmortem-ready record of when recovery was verified and by which rule run.

Who it's for

• On-call engineers who want "verified by rerunning the monitor" to be a step the tooling performs, not a habit.
• Incident managers tired of reopened tickets caused by fixes that did not actually fix the data.
• Data platform teams connecting Sifflet detections to a closed-loop incident process.

Why orchestrate this with Kestra

Sifflet detects and your incident tool tracks, but neither reruns the check when the fix lands. Kestra is the connective step: a webhook any tool can call, the rule run launched with credentials kept in secrets, the verdict pushed back into the channel where the incident lives, and an execution record for the postmortem. The recheck stops depending on whoever happens to be resolving the ticket.

Prerequisites

• The Sifflet rule ID of the monitor that fires for this class of incident.
• A Sifflet API key with permission to run rules.
• An incident tool able to POST to a URL when a fix is deployed.
• A Slack incoming webhook posting into the incident channel.

Secrets

• SIFFLET_API_KEY: Sifflet API bearer token used to run the rule.
• SIFFLET_BASE_URL: base URL of your Sifflet tenant API.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.
• SIFFLET_RECHECK_WEBHOOK_KEY: secret key guarding the webhook endpoint.

Quick start

1. Add the four secrets to your Kestra namespace.
2. Execute the flow manually with a real rule_id and confirm the Slack verdict arrives.
3. Point your incident tooling at the webhook URL, which includes the flow ID and the secret key, and fire a test event.

How to extend

• Clone the flow per incident class, each with the matching rule ID as the input default.
• Recheck a whole rule pack after large incidents by adding a ForEach, as shown in the Sifflet rule pack fan-out blueprint.
• Add retry on the rule task so a transient API blip does not fail the recheck during an active incident.
• Pass incident metadata in the webhook body and echo it in the Slack message through {{ trigger.body }} fields when the flow is webhook-triggered.

Links

• Sifflet plugin documentation
• Sifflet documentation