Kestra Flow Failure Rate Gate

A namespace whose flows start failing repeatedly - a credential rotated without updating a secret, an upstream API that changed shape, a dependency that stopped resolving - does not announce itself unless someone is actively watching the executions page. This blueprint uses Kestra's own plugin-kestra to count how many executions in a namespace ended FAILED over a lookback window, and alerts Slack the moment that count crosses a threshold you set, with the failure rate against total executions in the same window for context.

How it works

1. periodic_failure_check (io.kestra.plugin.core.trigger.Schedule) runs every hour. Shipped disabled so you can validate a manual run first.
2. count_recent_failures (io.kestra.plugin.kestra.executions.Count) counts executions in target_namespace with states: [FAILED] between now() - lookback_hours and now(), returning count.
3. count_total_executions runs the same query with no state filter, so the total in the same window is available for a rate calculation.
4. evaluate_failure_rate (io.kestra.plugin.core.flow.If) branches on count_recent_failures.count > max_failures_allowed.
   • then (AT_RISK): alert_failure_rate_breach posts the failed count, the threshold, and the failure rate (guarded against division by zero when the window had no other executions at all).
   • else (HEALTHY): log_failure_rate_healthy records a clean check.
5. log_audit_complete always runs last, printing both counts regardless of which branch fired.
6. The errors block alerts Slack separately if the flow itself fails outright - an unreachable Kestra API or bad credentials must not read as "no risk".

What you get

• A direct count from Kestra's own execution search API, the same data backing the Executions page, not a scrape or a third-party monitoring agent.
• A failure rate alongside the raw count, so "5 failures" reads differently against 6 total executions than against 600.
• A read-only design: both tasks only ever count - this flow cannot kill, resume, retry, or delete any execution.
• A final status log every run, breach or not, so the execution history doubles as a failure-rate trend for the namespace.

Who it's for

• Platform teams running Kestra who want a namespace-level failure-rate alert without building a custom dashboard query every time.
• Teams onboarding a new namespace who want an early, low-effort signal during the shakeout period before fully trusting it.
• Anyone evaluating plugin-kestra who wants a worked example of executions.Count used as a genuine threshold gate, distinct from this repo's existing kestra-flow-health-scorecard.yaml (a Python-script-driven statistical spike detector comparing recent duration/failure-rate against a rolling baseline) and tencent-qq-pipeline-digest.yaml (an unconditional reporting digest, no branch or alert).

Why orchestrate this with Kestra

A failure count pulled by hand from the Executions page answers one moment in time; it does not decide a cadence, compare the result against a threshold, or notify anyone. Kestra supplies the schedule, the threshold comparison as a first-class branch, and an execution history that shows exactly when a namespace's failure rate crossed the line - using Kestra to watch Kestra.

Prerequisites

• A reachable Kestra instance (the same one this flow runs on, or another one) with its API enabled.
• Credentials with permission to search executions via the API.
• A Slack incoming webhook for alerts.

Local testing: this flow talks to a Kestra API, so there is no separate Docker image to stand up beyond Kestra itself - point kestra_url (via secret, per this repo's own existing plugin-kestra convention) at any reachable Kestra instance, local or otherwise. Avoid pointing it at port 8080 only if that happens to collide with something else on the same host; 8080 is Kestra's own default UI/API port and is expected here, not avoided.

Secrets

• KESTRA_API_URL: base URL of the Kestra instance's API, used as kestraUrl on both Count tasks - for example http://localhost:8080.
• KESTRA_API_USERNAME / KESTRA_API_PASSWORD: credentials used by both Count tasks' auth block.
• SLACK_WEBHOOK_URL: Slack incoming webhook used by alert_failure_rate_breach and the errors block.
• In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded, and read back in flows with {{ secret('NAME') }} - for example SECRET_KESTRA_API_PASSWORD=$(echo -n '<your-local-password>' | base64). This keeps credentials out of the flow YAML but, per Kestra's own documentation, offers no encryption at rest or access control beyond the host environment; use the Enterprise secrets backend for stronger guarantees.

Inputs

• target_namespace (STRING, default company.team): namespace audited, exact match.
• lookback_hours (INT, default 24): window size.
• max_failures_allowed (INT, default 5): failure-count ceiling before AT_RISK.

Outputs

• outputs.count_recent_failures.count: FAILED execution count in the window, every run.
• outputs.count_total_executions.count: total execution count in the same window, every run.

Quick start

1. Point target_namespace at a namespace with some execution history, real or seeded by running a simple flow a few times, including at least one deliberate failure.
2. Add the KESTRA_API_URL, KESTRA_API_USERNAME, KESTRA_API_PASSWORD, and SLACK_WEBHOOK_URL secrets.
3. Run the flow manually with the defaults and confirm log_failure_rate_healthy fires if failures are at or below 5.
4. Lower max_failures_allowed below your actual failure count and re-run to confirm alert_failure_rate_breach fires with the correct rate.
5. Enable periodic_failure_check once you trust the check.

How to extend

• Loop over several target_namespace values with io.kestra.plugin.core.flow.Loop to run the same gate across every namespace a platform team owns in one flow.
• Add a flowId filter on both Count tasks (a real property on this task, unused here) to scope the gate to one specific flow instead of a whole namespace.
• Use io.kestra.plugin.kestra.executions.Query instead of Count to also list the failing executions' IDs in the Slack message, not just the count - Query's FetchOutput.rows wraps the full result list as a single element (verified from the plugin's own source), so unwrap it with outputs.x.rows | first before iterating.
• Pair this with io.kestra.plugin.kestra.executions.Kill on a separate, carefully-gated branch for executions stuck in a RUNNING state past an expected duration - a different risk than a burst of FAILED executions, and one this flow does not attempt to address.

Pitfalls

• This only counts FAILED executions, not WARNING, KILLED, or other non-SUCCESS terminal states. A namespace accumulating KILLED or other non-FAILED terminal states would not trip this gate - add additional states entries or a second Count task if those matter to you too.
• The failure rate in the Slack message is guarded against division by zero, not against a misleading small sample. 6 of 6 executions and 6 of 600 both read as percentages, but the first is a far weaker signal - read the raw totals alongside the rate, not the rate alone.
• target_namespace matches exactly, not as a prefix. Child namespaces (company.team.sub) are not included unless you loop over them separately - verified from the plugin's own Count source, the NAMESPACE filter is an EQUALS operation, not a prefix match.
• There is no remediation action in this flow by design. A burst of failures needs a human to diagnose the actual cause - an automated retry of whatever already failed risks compounding the problem rather than fixing it, which is why alert_failure_rate_breach only reports.
• This blueprint is UNTESTED against a live Kestra API. executions.Count's properties and output (namespaces, flowId, states, startDate, endDate, expression, count) are taken verbatim from the plugin's source on main, and the dateAdd(-inputs.x, 'HOURS') pattern is copied from this repo's own freshdesk-sla-breach-sentinel.yaml - but no second Kestra instance or engine run was exercised to execute this flow end to end.

Links

• plugin-kestra documentation
• plugin-kestra source (executions/Count.java)
• Kestra secrets (OSS vs Enterprise)