Log icon
Sleep icon
Subflow icon
Flow icon
ExecutionStatus icon
ExecutionNamespace icon
ExecutionLabels icon

Auto Resubmit Failed Executions with Loop Protection

Kestra blueprint that detects FAILED or WARNING executions with a Flow trigger and re-executes them as subflows, using execution labels to prevent retry loops.

Categories
Core

Recover from transient failures without a human clicking restart. This Kestra blueprint listens for executions that end in FAILED or WARNING anywhere under a namespace prefix, waits a short grace period, and re-executes the failed flow by launching it as a subflow with the same flow id and namespace read from the trigger context. The critical piece is loop protection: every resubmitted execution is stamped with an implicit: true label, and the trigger only matches executions labeled implicit: false, so a resubmission that fails again can never trigger itself into an infinite retry storm.

How it works

  1. The listen trigger (io.kestra.plugin.core.trigger.Flow) watches execution state changes. An io.kestra.plugin.core.condition.ExecutionStatus condition matches terminal FAILED and WARNING states, and io.kestra.plugin.core.condition.ExecutionNamespace with prefix: true scopes it to the company tree.
  2. The io.kestra.plugin.core.condition.ExecutionLabels condition requires the label implicit: false on the failed execution. Flows that opt in to auto resubmission carry this label; resubmitted runs carry implicit: true and are therefore ignored, breaking the loop after one retry.
  3. The log_failure task (io.kestra.plugin.core.log.Log) records which flow and execution are being resubmitted, along with the triggering execution's labels.
  4. The sleep task (io.kestra.plugin.core.flow.Sleep) waits PT3S as a simple backoff before retrying (tune this to your failure modes).
  5. The retry_subflow task (io.kestra.plugin.core.flow.Subflow) starts a fresh execution of {{ trigger.flowId }} in {{ trigger.namespace }} with wait: true and transmitFailed: false, so this alerting flow itself stays green even when the retry fails, and stamps the run with implicit: true.
  6. The trigger ships disabled: true as a safety default: review the conditions and scope before switching it on.

What you get

  • Hands free resubmission of failed executions across a whole namespace tree.
  • Deterministic loop protection through execution labels, one retry per failure, never a storm.
  • A grace period before retrying, tunable per environment.
  • A resubmission run whose own status stays clean thanks to "transmitFailed: false".
  • Full audit trail linking the failed execution, the resubmitter, and the retried run.

Who it's for

  • Platform teams whose pipelines fail on transient causes such as timeouts or brief outages.
  • On call engineers tired of manually restarting the same nightly jobs.
  • Teams composing this with failure alerting flows for a detect, notify, retry loop.

Why orchestrate this with Kestra

Task level retries only help while the execution is still alive; once a run has failed terminally, something outside it must decide to try again. Kestra's Flow trigger makes that decision declarative: conditions define which failures qualify, Subflow re-executes the target generically for any flow id in the namespace, and execution labels carry the retry state with no external database. The whole recovery policy is one small flow you can version, review, and disable instantly.

Prerequisites

  • Flows you want auto resubmitted must set the execution label implicit: false so they match the ExecutionLabels condition.
  • Review and adjust the namespace prefix, then set disabled: false on the trigger to activate it.
  • Target flows should be idempotent, since they will be re-run from the start.

Secrets

  • None required. Add a Slack webhook secret if you extend the flow with notifications.

Quick start

  1. Deploy the flow and adjust namespace: company to the tree you want covered.
  2. Add the label implicit: false to a test flow and make it fail on purpose.
  3. Set disabled: false on the listen trigger.
  4. Watch the failed execution get picked up, logged, and re-executed once, and confirm the retried run (labeled implicit: true) does not re-trigger.

How to extend

  • Increase the Sleep duration or replace it with an exponential backoff via multiple conditions.
  • Add an io.kestra.plugin.core.condition.ExecutionFlow condition to limit resubmission to specific critical flows.
  • Notify Slack before and after the retry with io.kestra.plugin.slack.notifications.SlackIncomingWebhook.
  • Set transmitFailed: true and add an errors branch to escalate when the retry also fails.
  • Track retry counts in the KV store to cap total attempts per flow per day.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.