SquadcastIncomingWebhook icon
SlackIncomingWebhook icon
Webhook icon

Auto-Resolve Squadcast Incidents on Recovery

Auto-resolve Squadcast incidents when systems recover. A Kestra webhook receives the recovery signal and sends status resolve with the original event id.

Categories
BusinessInfrastructure

Half of incident hygiene is closing pages, and it is the half that gets forgotten. Squadcast's incoming webhook correlates calls by event id, an alert sent with status: trigger and an event_id opens an incident, and a later call with status: resolve and the same event_id closes it. This blueprint automates the second half. A Kestra webhook trigger receives a recovery signal from the healing system or its monitor, sends the resolve payload through io.kestra.plugin.squadcast.SquadcastIncomingWebhook, and posts a Slack note, so incidents opened by your pager flows close themselves the moment recovery is confirmed.

How it works

  1. on_recovery (io.kestra.plugin.core.trigger.Webhook) exposes an endpoint guarded by a secret key. The recovering service, a synthetic check, or another Kestra flow posts a JSON body containing event_id once health returns.
  2. resolve_incident posts a two-field payload, status: resolve and event_id, matching the resolve semantics documented on the plugin page. The expression {{ trigger.body.event_id ?? inputs.event_id }} reads the live webhook body in production and falls back to the event_id input for manual runs.
  3. note_resolution leaves a Slack line naming the resolved event id, so the recovery is visible in the channel without a human touching Squadcast.
  4. The errors block reports to Slack when the resolve call fails, because a resolved system with an open page erodes trust in the pager in both directions.

What you get

  • Incidents that close when the system heals, not when someone remembers.
  • Event-id correlation with your trigger flows, the flow failure pager and heartbeat pager blueprints both set an event_id this flow can resolve.
  • A manual fallback path via the input, useful for closing an incident from the Kestra UI.
  • An explicit alert when auto-resolution itself fails, so stale pages get human attention.

Who it's for

  • On-call teams drowning in incidents that recovered on their own hours ago.
  • Platform engineers wiring trigger and resolve into the same event lifecycle.
  • Anyone measuring MTTR who wants resolution timestamps to reflect actual recovery.

Why orchestrate this with Kestra

Auto-resolution needs an authenticated HTTP entry point, payload templating with a safe manual fallback, a pager integration, and an audit trail of every close. Kestra's webhook trigger, Pebble expressions, and execution history provide all of it in a dozen lines of YAML, and the same pattern extends to any system that reports its own recovery.

Prerequisites

  • A Squadcast Incoming Webhook (API) integration on the target service.
  • Pager flows that set a deterministic event_id when they trigger incidents.
  • A Slack incoming webhook for notes and failure reports.

Secrets

  • SQUADCAST_WEBHOOK_URL: the full Squadcast incoming webhook endpoint, including its token.
  • SLACK_WEBHOOK_URL: Slack incoming webhook URL.
  • RESOLVE_WEBHOOK_KEY: the key guarding the Kestra webhook endpoint.

Quick start

  1. Add the three secrets to your namespace.
  2. Trigger a test incident with a known event id, for example from the heartbeat silence pager blueprint.
  3. POST {"event_id": "your-event-id"} to the flow's webhook URL and confirm the Squadcast incident resolves and the Slack note appears.
  4. Wire your service's recovery hook or synthetic monitor to the webhook endpoint.

How to extend

  • Have the heartbeat pager's healthy branch call this webhook, so the same flow that detected silence confirms recovery.
  • Add a verification step before resolving, probe the recovered system once more and only resolve on a 200.
  • Fan the recovery signal out to a status page or ticket system alongside the Squadcast resolve.
  • Log resolved event ids to a KV store to build a recovery-time dataset for MTTR reporting.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.