SLO Error Budget Gate Backed by Prometheus

An error budget only changes behavior if something enforces it. This blueprint turns the budget into a gate with a machine-readable verdict. Each day, io.kestra.plugin.prometheus.Query computes the ratio of 5xx requests to all requests over the last hour with fetchType: FETCH_ONE. An If task compares the scalar against a 0.1 percent threshold. Above it, the flow alerts Slack and fails through io.kestra.plugin.core.execution.Fail, so any pipeline gated on this flow's success, deploys included, stops. Below it, the flow logs the healthy reading and exits green.

How it works

1. check_error_rate (io.kestra.plugin.prometheus.Query, fetchType: FETCH_ONE) evaluates the PromQL ratio and exposes the single row as {{ outputs.check_error_rate.row }}.
2. budget_gate (io.kestra.plugin.core.flow.If) compares the row value, cast with the number filter, against 0.001.
3. On a breach, alert_budget_burn posts the measured rate to Slack, then fail_gate fails the execution with an explicit errorMessage carrying the number.
4. Within budget, budget_ok logs the reading, turning the execution history into an SLO audit trail.
5. The errors block posts a Slack alert whenever the execution fails, covering both a genuine breach and a broken query.
6. A disabled-by-default Schedule trigger runs the gate daily at 06:00, before the deploy window.

What you get

• A pass or fail verdict on the error budget that other automation can depend on.
• The measured rate embedded in the alert, the failure message, and the log, never a bare threshold alarm.
• An execution history that documents every budget check for reviews and postmortems.
• A threshold and window you change in two places, the PromQL range and the If condition.

Who it's for

• SRE teams formalizing error budget policy into something executable.
• Platform teams who want deploy pipelines to respect reliability automatically.
• Engineering leads who need an auditable record that the budget was checked.

Why orchestrate this with Kestra

A Prometheus alert can tell you the budget burned; it cannot stop a deploy. Because this gate is a flow with a real terminal state, other flows can require its success, CI systems can poll its outcome, and the whole policy lives in version-controlled YAML with a complete history of every decision it made.

Prerequisites

• A Prometheus server scraping a service that exposes http_requests_total with a status label, or adjust the PromQL to your metric names.
• Enough traffic in the evaluation window for the ratio to be defined; a service with zero requests returns an empty result.
• A Slack incoming webhook for alerts.
• The Query task supports HTTP basic auth through its username and password properties if Prometheus sits behind one.

Secrets

• PROMETHEUS_URL: base URL of the Prometheus server, e.g. http://prometheus:9090.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the PROMETHEUS_URL and SLACK_WEBHOOK_URL secrets to your Kestra namespace.
2. Adjust the PromQL to your metric and the threshold to your SLO, in both the condition and the messages.
3. Execute the flow and verify the green path logs the measured rate.
4. Set disabled: false on the daily trigger.

How to extend

• Gate deploys directly by calling this flow with io.kestra.plugin.core.flow.Subflow at the start of a deploy pipeline.
• Run per-service gates with io.kestra.plugin.core.flow.Loop over a list of label selectors.
• Use a longer burn window, such as [6h] alongside [1h], to implement multi-window burn rate alerts.
• Push the verdict back to Prometheus with the Push task so dashboards show gate outcomes over time.

Links

• Prometheus plugin documentation
• Google SRE error budget documentation