ScheduleMonitor icon
If icon
SlackIncomingWebhook icon
Log icon

Kestra Schedule Trigger Health Sentinel

Detect overdue, stuck, or disabled Kestra SCHEDULE triggers via the API and alert Slack with the affected list.

Categories
CoreInfrastructure

A SCHEDULE trigger that stops firing - because its flow was disabled, a worker fell behind, or the Kestra API silently stopped evaluating it - does not announce itself. The flows depending on it just quietly stop running, and nobody notices until something downstream that expected a fresh output does not get one. This blueprint uses Kestra's own plugin-kestra to poll the trigger search API directly for exactly this condition and alert Slack the moment it is detected, with the full list of affected triggers.

How it works

  1. monitor_schedule_health (io.kestra.plugin.kestra.triggers.ScheduleMonitor) polls every 5 minutes. It flags a SCHEDULE trigger when: it is disabled (because includeDisabled: true); it has gone longer than maxExecutionInterval (24h) since its last run; it has no next execution date at all; or the current time is past its next execution date plus allowedDelay (10 minutes). Shipped disabled.
  2. Critically, ScheduleMonitor's own evaluate() returns no execution whatsoever when it finds zero unhealthy triggers (verified from the plugin's source: if (result.getData().isEmpty()) return Optional.empty();). That means this flow's own execution history only ever contains runs where something was already wrong - there is no corresponding "all clear" execution to point to, and no HEALTHY branch to write inside the flow.
  3. evaluate_drift_severity (io.kestra.plugin.core.flow.If) branches on trigger.data | length >= 3 instead of a health/unhealthy split, since every execution is already unhealthy by definition.
    • then (WIDESPREAD): alert_widespread_schedule_drift posts the count and the full namespace/flowId/triggerId list, flagging a likely shared root cause.
    • else (ISOLATED): alert_isolated_schedule_drift posts the same detail with less urgent wording.
  4. log_audit_complete always runs last, logging the same count and list independent of which branch fired.
  5. The errors block alerts Slack separately if the flow itself fails - an unreachable Kestra API must not be mistaken for "no unhealthy triggers".

What you get

  • Detection sourced from Kestra's own trigger search API, the same data backing the Triggers page, not a log scrape or a separate monitoring agent.
  • Coverage of three distinct failure modes in one check: overdue past an allowed delay, missing a next run entirely, and (optionally) simply disabled.
  • A staleness floor (maxExecutionInterval) independent of the next-run calculation, so a trigger that still reports a plausible next run but has not actually fired in a day is still caught.
  • Severity-aware alerting that distinguishes a single flaky trigger from a fleet-wide problem, without needing a second monitoring system to correlate them.

Who it's for

  • Platform teams running Kestra who want to know when a SCHEDULE trigger silently stops firing, before someone downstream notices a stale output.
  • Teams migrating or reorganizing namespaces who want a safety net for accidentally-disabled triggers during the transition.
  • Anyone evaluating plugin-kestra who wants a worked example of triggers.ScheduleMonitor, distinct from this repo's existing kestra-flow-failure-rate-gate.yaml (counts FAILED executions after the fact) and kestra-flow-health-scorecard.yaml (a statistical duration/failure-rate spike detector) - this flow instead watches whether a trigger is firing at all, independent of whether its executions succeed.

Why orchestrate this with Kestra

Checking whether a schedule "looks right" by eye does not scale past a handful of flows, and does not catch a trigger that silently stopped evaluating at all. Kestra supplies the recurring poll, a direct API call into its own trigger state, and an execution history that records exactly when and how many triggers went unhealthy - using Kestra to watch Kestra.

Prerequisites

  • A reachable Kestra instance (the same one this flow runs on, or another one) with its API enabled and at least one SCHEDULE trigger to monitor.
  • Credentials with permission to search triggers via the API.
  • A Slack incoming webhook for alerts.

Local testing: this flow talks to a Kestra API, so there is no separate Docker image to stand up beyond Kestra itself - point kestraUrl (via secret, per this repo's own existing plugin-kestra convention) at any reachable Kestra instance, local or otherwise. 8080 is Kestra's own default UI/API port and is expected here, not avoided.

Secrets

  • KESTRA_API_URL: base URL of the Kestra instance's API, used as kestraUrl on the trigger.
  • KESTRA_API_USERNAME / KESTRA_API_PASSWORD: credentials used by the trigger's auth block.
  • SLACK_WEBHOOK_URL: Slack incoming webhook used by both alert tasks and the errors block.
  • In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded, and read back in flows with {{ secret('NAME') }} - for example SECRET_KESTRA_API_PASSWORD=$(echo -n '<your-local-password>' | base64). This keeps credentials out of the flow YAML but, per Kestra's own documentation, offers no encryption at rest or access control beyond the host environment; use the Enterprise secrets backend for stronger guarantees.

Inputs

  • None. Trigger properties cannot reference inputs.* - they are evaluated before any execution or input context exists - so scope and thresholds (namespace, flowId, includeDisabled, allowedDelay, maxExecutionInterval, interval) and the severity threshold (3, inside evaluate_drift_severity) are edited directly in this file.

Outputs

  • trigger.data: the list of unhealthy TriggerInfo records (namespace, flowId, triggerId, lastExecution, expectedNext) for the execution that fired, every run.

Quick start

  1. Add the KESTRA_API_URL, KESTRA_API_USERNAME, KESTRA_API_PASSWORD, and SLACK_WEBHOOK_URL secrets.
  2. Disable one existing SCHEDULE trigger in a test namespace, or set allowedDelay to something very small (for example PT1S) against a trigger you know is slightly behind, so there is a real unhealthy trigger to find.
  3. Narrow the blast radius first by setting namespace on monitor_schedule_health to that test namespace's prefix (left unset here, which scans every namespace).
  4. Run the flow manually (polling triggers can also be run once on demand) and confirm alert_isolated_schedule_drift fires with that trigger listed.
  5. Enable monitor_schedule_health once you trust the check, and widen or remove the namespace filter to cover the namespaces you actually want watched.

How to extend

  • Pair this with io.kestra.plugin.kestra.triggers.Toggle on a tightly-gated, human-approved branch to re-enable a trigger found disabled by mistake - not baked into this flow's core alerting, since auto-re-enabling a trigger that was deliberately disabled could restart something that was intentionally paused.
  • Set flowId on the trigger to narrow monitoring to a single flow's schedule instead of every SCHEDULE trigger in a namespace.
  • Drop includeDisabled to false if a dedicated flow already tracks intentional disables, and this gate should only report triggers that are enabled but unhealthy.
  • Loop over the WIDESPREAD branch's affected list with io.kestra.plugin.core.flow.Loop and fromJson(item.value) (this repo's standard pattern for iterating a jq-filtered list) to file one ticket per affected flow instead of a single combined Slack message.

Pitfalls

  • This flow has no HEALTHY branch and no "all clear" execution. ScheduleMonitor itself only ever generates an execution when trigger.data is non-empty (verified from the plugin's own evaluate() source) - the complete absence of executions over your expected poll interval is the healthy signal, not a log line inside this flow. Monitor this flow's own execution frequency, not just its outcomes.
  • namespace is unset here, scanning every namespace on the instance. On a large Kestra instance this can be a wide net - set namespace to a prefix filter deliberately rather than leaving it unscoped in production.
  • allowedDelay and maxExecutionInterval are independent checks, not layered thresholds. A trigger can trip maxExecutionInterval (stale since its last actual run) while still reporting a next execution date within allowedDelay, or vice versa - both are evaluated, verified directly from runChecks()'s source.
  • The severity threshold (3) is a literal value in this file, not a tunable input, because trigger-level properties cannot read inputs.* - adjust it directly in evaluate_drift_severity's condition if 3 is not the right split for your fleet size.
  • This blueprint is UNTESTED against a live Kestra API. ScheduleMonitor's properties (interval, namespace, flowId, includeDisabled, allowedDelay, maxExecutionInterval) and output (data with namespace/flowId/triggerId/lastExecution/expectedNext) are taken verbatim from the plugin's source on main, and the trigger.data | jq(...) | first pattern mirrors this repo's own corrected convention for formatting a jq-filtered list into a Slack message - but no live instance was polled end to end to confirm evaluate()'s behavior.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.