id: kestra-schedule-trigger-health-sentinel
namespace: company.team
description: |
Poll Kestra's own trigger search API for SCHEDULE triggers that are overdue,
missing their next execution date, or - optionally - sitting disabled, and
alert Slack with the exact list the moment any are found. Unlike a normal
health gate, this flow's own trigger only ever fires when something is
already unhealthy, so there is no HEALTHY branch inside the flow itself -
severity is judged by how many triggers are affected at once. Read-only:
this flow never enables, disables, or otherwise touches any trigger.
triggers:
- id: monitor_schedule_health
type: io.kestra.plugin.kestra.triggers.ScheduleMonitor
description: >
Polls every 5 minutes (plugin default is 60s - widened here to avoid
hammering the API) for SCHEDULE triggers that are overdue past
allowedDelay, have no next execution date at all, have gone longer than
maxExecutionInterval since their last run, or - because includeDisabled is
true here - are simply disabled. Shipped disabled; enable once
kestraUrl/auth point at a real Kestra instance and a manual check of
runChecks' behavior has been verified. Trigger properties cannot read flow
inputs - they evaluate before an execution context exists - so scope
(namespace/flowId) and thresholds are edited directly below, not via an
inputs: block.
disabled: true
interval: PT5M
includeDisabled: true
allowedDelay: PT10M
maxExecutionInterval: PT24H
kestraUrl: "{{ secret('KESTRA_API_URL') }}"
auth:
username: "{{ secret('KESTRA_API_USERNAME') }}"
password: "{{ secret('KESTRA_API_PASSWORD') }}"
tasks:
- id: evaluate_drift_severity
type: io.kestra.plugin.core.flow.If
description: >
There is no "else branch is healthy" here - trigger.data is only ever
non-empty, because ScheduleMonitor's own evaluate() returns no execution
at all when it finds zero unhealthy triggers (verified from the plugin's
own source). Both branches alert; they differ only in urgency wording
based on how many triggers are affected at once. 3 is a literal threshold
edited directly in this file, not a flow input, for the same reason the
trigger block above has none.
condition: "{{ trigger.data | length >= 3 }}"
then:
- id: alert_widespread_schedule_drift
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
description: 3 or more SCHEDULE triggers unhealthy at once - likely a shared
cause (API outage, worker starvation, a bad deploy) rather than one
flaky flow.
url: "{{ secret('SLACK_WEBHOOK_URL') }}"
payload: |
{
"text": "Kestra schedule trigger health sentinel: WIDESPREAD. {{ trigger.data | length }} SCHEDULE trigger(s) are overdue, missing a next run, stale past maxExecutionInterval, or disabled: {{ trigger.data | jq('[.[] | .namespace + \"/\" + .flowId + \"/\" + .triggerId] | join(\"; \")') | first }}. This many at once suggests a shared root cause, not one flaky flow - check worker health and the Kestra API before triaging triggers one by one. Execution {{ execution.id }}."
}
else:
- id: alert_isolated_schedule_drift
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
description: Fewer than 3 unhealthy triggers - still real, but treated as
isolated cases to triage individually.
url: "{{ secret('SLACK_WEBHOOK_URL') }}"
payload: |
{
"text": "Kestra schedule trigger health sentinel: ISOLATED. {{ trigger.data | length }} SCHEDULE trigger(s) are overdue, missing a next run, stale past maxExecutionInterval, or disabled: {{ trigger.data | jq('[.[] | .namespace + \"/\" + .flowId + \"/\" + .triggerId] | join(\"; \")') | first }}. Execution {{ execution.id }}."
}
- id: log_audit_complete
type: io.kestra.plugin.core.log.Log
description: Final status line every time this flow runs - which, by design, is
only ever when at least one trigger is unhealthy.
message: "Schedule trigger health check found {{ trigger.data | length }}
unhealthy trigger(s): {{ trigger.data | jq('[.[] | .namespace + \"/\" +
.flowId + \"/\" + .triggerId] | join(\"; \")') | first }}."
errors:
- id: alert_execution_failure
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
description: Alert when the flow itself fails outright - an unreachable Kestra
API or bad credentials, distinct from the triggers it is trying to report
on being unhealthy.
url: "{{ secret('SLACK_WEBHOOK_URL') }}"
payload: |
{
"text": "Kestra schedule trigger health sentinel FAILED in execution {{ execution.id }}. The trigger search call did not complete; investigate the Kestra API connection before trusting the last known trigger health status."
}