AWS CloudFormation Stuck Stack Failure and Rollback Sentinel

A CloudFormation stack that fails to create or update rolls back to a terminal state like ROLLBACK_COMPLETE or UPDATE_ROLLBACK_FAILED and then simply sits there - no new deployment can proceed against it until someone manually deletes or fixes it, and nothing in CloudFormation itself proactively announces that a stack is stuck. This blueprint reads a stack's own StackStatus and StackStatusReason on a schedule and alerts the moment it lands in one of CloudFormation's own documented failure or rollback states.

This blueprint was created by Mohit Jeswani.

How it works

1. periodic_stack_check (io.kestra.plugin.core.trigger.Schedule) runs every 30 minutes, explicitly passing aws_region, stack_name, and aws_endpoint_override through its own inputs: block so the flow is self-contained when the trigger fires. Shipped disabled so a manual run can be verified first.
2. probe_stack_status (io.kestra.plugin.aws.cli.AwsCLI) runs aws cloudformation describe-stacks --query 'Stacks[0]' and captures the result's fields (StackStatus, StackStatusReason, and every other field CloudFormation returns) as named task outputs via the plugin's own documented output-echo convention.
3. evaluate_stack_health (io.kestra.plugin.core.flow.If) branches on whether StackStatus is one of CloudFormation's own documented terminal failure or rollback values.
   • then (NEEDS_ATTENTION): alert_stack_needs_attention reports the status and StackStatusReason to Slack.
   • else (HEALTHY or IN_PROGRESS): log_stack_healthy records the current status.
4. log_audit_complete always runs last, printing the stack's status regardless of which branch fired.
5. The errors block alerts Slack separately if the flow itself fails outright - an unreachable endpoint, a missing stack, or bad credentials must not read as either handled state.

What you get

• A HEALTHY/IN_PROGRESS/NEEDS_ATTENTION answer sourced directly from CloudFormation's own StackStatus, not inferred from a deploy pipeline's own exit code, which never sees a stack that failed and rolled back cleanly.
• The stack's own StackStatusReason delivered straight to Slack, instead of requiring someone to open the CloudFormation console to read why a stack is stuck.
• A worked example of io.kestra.plugin.aws.cli.AwsCLI driving a read-only audit query with --query and capturing structured output via the plugin's own output-echo convention, not just running a provisioning command.
• A flow that is genuinely runnable against LocalStack via endpointOverride, with no AWS account or CloudFormation drift-detection feature (which LocalStack does not implement, even on paid tiers) required.

Who it's for

• Platform teams running CloudFormation who need an early signal when a stack lands in ROLLBACK_COMPLETE or a _FAILED state and blocks the next deployment.
• On-call engineers who want one Slack alert with the real StackStatusReason instead of opening the CloudFormation console to find out why a stack is stuck.
• Anyone evaluating the AWS plugin who wants a worked example of AwsCLI used for a read-only governance probe, paired with its documented structured-output convention.

Why orchestrate this with Kestra

CloudFormation itself has no scheduler and no "this stack has been stuck for a while" alert - a failed rollback sits silently until the next deploy attempt trips over it. Kestra supplies the schedule, the HEALTHY/IN_PROGRESS/NEEDS_ATTENTION classification as a first-class branch, and an execution history that shows exactly when a stack crossed into a failure state and what CloudFormation's own reason string said.

Prerequisites

• An AWS account or a local LocalStack instance with at least one CloudFormation stack created (update stack_name).
• AWS CLI credentials with cloudformation:DescribeStacks permission.
• A Slack incoming webhook for alerts.

Local testing (LocalStack, no AWS account needed):

docker run -d --name localstack -p 4566:4566 localstack/localstack:latest
docker network connect YOUR_KESTRA_NETWORK localstack
aws --endpoint-url=http://localhost:4566 cloudformation create-stack --stack-name test-stack --template-body '{"Resources":{"Bucket":{"Type":"AWS::S3::Bucket"}}}'

Run docker network ls first to find the active Kestra network name. Port 4566 is LocalStack's single edge port for every emulated AWS service and does not collide with Kestra's own UI on port 8080. From inside the Kestra worker's Docker network, set aws_endpoint_override to http://localstack:4566 instead of http://localhost:4566. describe-stacks (used by this blueprint) is implemented by LocalStack's free Community edition; CloudFormation drift detection (detect-stack-drift) is not implemented by LocalStack at all, which is why this blueprint audits StackStatus instead.

Secrets

• AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY: credentials for io.kestra.plugin.aws.cli.AwsCLI. Against LocalStack, any non-empty placeholder value works (LocalStack Community does not validate credentials) - the well-known convention is literally the strings test / test.
• SLACK_WEBHOOK_URL: Slack incoming webhook used by alert_stack_needs_attention and the errors block.
• In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded - for example SECRET_AWS_ACCESS_KEY_ID=$(echo -n 'test' | base64) - and read back in flows with {{ secret('NAME') }}.

Inputs

Input	Type	Default	Purpose
aws_region	String	us-east-1	Region passed to AwsCLI as AWS_DEFAULT_REGION
stack_name	String	<your-cloudformation-stack-name>	Name or ARN of the CloudFormation stack to audit
aws_endpoint_override	String	http://localhost:4566	LocalStack endpoint override; clear for production AWS

Outputs

• outputs.probe_stack_status.vars.StackStatus / StackStatusReason: the two fields this blueprint classifies on.
• outputs.probe_stack_status.vars.*: every other field CloudFormation's describe-stacks returns for the stack (StackId, CreationTime, Outputs, Parameters, DriftInformation, and more) is captured the same way, even though only the two above are read here.

Quick start

1. Start LocalStack and create test-stack with the commands above.
2. Add the AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY (test / test against LocalStack) and SLACK_WEBHOOK_URL secrets.
3. Run the flow manually with stack_name: test-stack and confirm log_stack_healthy fires with CREATE_COMPLETE.
4. Create a stack that deliberately fails (for example, reference a resource type or property that cannot be created) so it reaches ROLLBACK_COMPLETE, then re-run to confirm alert_stack_needs_attention fires with the real StackStatusReason.
5. Clear aws_endpoint_override and point this at a real AWS account and stack once you trust the check; enable periodic_stack_check once you trust the cadence.

How to extend

• Loop over several stack_name values with io.kestra.plugin.core.flow.Loop to audit every critical stack in an account in one run.
• On the NEEDS_ATTENTION branch, add a second io.kestra.plugin.aws.cli.AwsCLI task running aws cloudformation delete-stack behind an explicit dry_run/approval gate for stacks stuck in ROLLBACK_COMPLETE (CloudFormation's own documented recovery path for that specific state - a stack in ROLLBACK_COMPLETE can only be deleted, never updated again).
• Once running against real AWS (not LocalStack), add a second, longer-interval flow using detect-stack-drift / describe-stack-drift-detection-status for configuration drift, since that governance angle is real but not exercisable locally.

Pitfalls

• LocalStack does not implement CloudFormation drift detection at all, not even on paid tiers - detect-stack-drift and describe-stack-drift-detection-status are absent from its supported operations. That is specifically why this blueprint audits StackStatus (which LocalStack's describe-stacks does support) instead of drift.
• aws_endpoint_override always sets AWS_ENDPOINT_URL, even when empty. This blueprint defaults it to a LocalStack URL for local testing; for production AWS, clear the input value rather than leaving it pointed at a local address, since an unreachable AWS_ENDPOINT_URL fails every call.
• The output-capture convention (tr -d '\n' | xargs -0 -I {} echo '::{"outputs":{}}::') is copied from io.kestra.plugin.aws.cli.AwsCLI's own Javadoc example, but with spaces no longer stripped. The plugin's own example additionally does tr -d ' \n', which would corrupt StackStatusReason's human-readable sentence by deleting every space in it.
• A stack in ROLLBACK_COMPLETE cannot be updated, only deleted, per CloudFormation's own documented behavior - log_stack_healthy/alert_stack_needs_attention only report the state; this blueprint does not attempt that recovery itself (see How to extend).
• This blueprint is UNTESTED against a live AWS account or LocalStack container. The StackStatus enum values and the StackStatusReason field are taken verbatim from AWS CloudFormation's own Stack API reference, and LocalStack's drift-detection gap is taken from LocalStack's own published supported-operations documentation - but no Docker container or Kestra engine was run to execute this flow end to end.

Links

• Kestra AWS plugin documentation
• AWS CloudFormation Stack API reference (StackStatus, StackStatusReason)
• LocalStack CloudFormation service coverage
• Kestra secrets (OSS vs Enterprise)