Azure Table Storage Stuck Entity Status Sentinel

A background job queue, sync worker, or device registry built on Azure Table Storage can leave entities stuck on a failed status forever - Table Storage itself has no scheduler and no "alert me if entities match this filter" primitive. This blueprint runs a server-side OData filter for a specific stuck-status value and alerts Slack with the count the moment any exist.

How it works

1. periodic_stuck_entity_check (io.kestra.plugin.core.trigger.Schedule) runs every 15 minutes, explicitly passing storage_account_endpoint, table_name, status_field, stuck_status_value, and max_entities_to_check through its own inputs: block so the flow is self-contained when the trigger fires. Shipped disabled so a manual run can be verified first.
2. probe_stuck_entities (io.kestra.plugin.azure.storage.table.List) runs an OData eq filter on status_field server-side, capped at max_entities_to_check, returning count.
3. evaluate_stuck_entities (io.kestra.plugin.core.flow.If) branches on whether count > 0.
   • then (NEEDS_ATTENTION): alert_stuck_entities reports the count to Slack.
   • else (HEALTHY): log_table_healthy records a clean check.
4. log_audit_complete always runs last, printing the stuck count regardless of which branch fired.
5. The errors block alerts Slack separately if the flow itself fails outright - an unreachable storage account or bad credentials must not read as HEALTHY.

What you get

• A HEALTHY/NEEDS_ATTENTION answer computed server-side by Azure Table Storage's own OData filter, not by downloading every entity and filtering client-side.
• A bounded query (maxFiles) so a large stuck-entity backlog cannot turn a routine health check into an expensive, unbounded read.
• A worked example of io.kestra.plugin.azure.storage.table.List's filter property, which every example in the plugin's own documentation leaves unused.
• A sentinel that is genuinely testable fully locally with zero Azure account, using Azurite - Microsoft's own open-source Storage emulator - and its publicly documented, well-known connection string.

Who it's for

• Teams running job queues, sync workers, or device registries on Azure Table Storage who need an early signal when entities start piling up in a failed state.
• On-call engineers who want one Slack alert with the exact stuck count instead of opening the Azure portal to run the same filter by hand.
• Anyone evaluating the Azure Storage plugin who wants a worked example of List's OData filter used as a governance probe rather than a bulk export step.

Why orchestrate this with Kestra

Table Storage itself has no concept of a schedule or a "notify me when this filter matches" trigger - checking requires either instrumenting every writer with its own alerting or someone remembering to run the query by hand. Kestra supplies the schedule, the HEALTHY/NEEDS_ATTENTION classification as a first-class branch, and an execution history that shows exactly when a backlog started and how large it grew.

Prerequisites

• An Azure Storage account with a table containing entities that carry a status property (update storage_account_endpoint, table_name, status_field), or Azurite for local testing.
• A connection string, SAS token, or shared key with read access to that table.
• A Slack incoming webhook for alerts.

Local testing (Azurite, no Azure account needed):

docker run -d --name azurite -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite
docker network connect YOUR_KESTRA_NETWORK azurite

Run docker network ls first to find the active Kestra network name. Azurite's table service listens on port 10002 by default, which does not collide with Kestra's own UI on port 8080. Set storage_account_endpoint to http://127.0.0.1:10002/devstoreaccount1 (or http://azurite:10002/devstoreaccount1 from inside the Kestra worker's Docker network), and set AZURE_STORAGE_CONNECTION_STRING to Azurite's own publicly documented, fixed well-known connection string (AccountName=devstoreaccount1 with its standard published key) - this is not a real secret, it is the same public value every Azurite instance ships with. Create the table and a test entity with the az CLI or any Table Storage SDK pointed at that same endpoint before running the flow.

Secrets

• AZURE_STORAGE_CONNECTION_STRING: connection string used by io.kestra.plugin.azure.storage.table.List. Against Azurite, this is the public well-known value described above, not a secret that needs protecting.
• SLACK_WEBHOOK_URL: Slack incoming webhook used by alert_stuck_entities and the errors block.
• In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded - for example SECRET_SLACK_WEBHOOK_URL=$(echo -n '<your-webhook-url>' | base64) - and read back in flows with {{ secret('NAME') }}.

Inputs

• storage_account_endpoint (STRING, default https://<your-account>.table.core.windows.net, must be replaced): Table Storage endpoint.
• table_name (STRING, default syncjobs): table to audit.
• status_field (STRING, default Status): property compared with eq.
• stuck_status_value (STRING, default FAILED): value identifying a stuck entity.
• max_entities_to_check (INT, default 100): cap on entities returned per run.

Outputs

• outputs.probe_stuck_entities.count: number of matching entities found this run.
• outputs.probe_stuck_entities.uri: Kestra storage URI of the full matching entity list, in case triage needs more than the count.

Quick start

1. Start Azurite with the command above and create a syncjobs table with one entity whose Status is not FAILED.
2. Add the AZURE_STORAGE_CONNECTION_STRING (Azurite's public well-known value) and SLACK_WEBHOOK_URL secrets.
3. Run the flow manually and confirm log_table_healthy fires.
4. Add (or update) an entity with Status set to FAILED and re-run to confirm alert_stuck_entities fires with a count of at least 1.
5. Point this at a real storage account and enable periodic_stuck_entity_check once you trust the check.

How to extend

• Loop over several table_name / stuck_status_value combinations with io.kestra.plugin.core.flow.Loop to audit multiple job queues' stuck-entity states in one run.
• Narrow the OData filter further with a PartitionKey eq '...' clause to scope the check to one tenant or device family within a shared table.
• Add a second, separate remediation flow that re-queues or deletes confirmed stuck entities behind an explicit approval gate - deliberately kept out of this sentinel so the monitoring path never modifies data.

Pitfalls

• Azure Table Storage partitions have no documented limit on entity count - the real, documented constraint is throughput (2,000 requests per second per partition, per Microsoft's own scalability targets). A row-count-based "partition is too big" check would measure the wrong thing entirely, which is why this blueprint checks entity status, not partition size, as its governance signal.
• The OData filter string is case-sensitive and must match the entity's actual stored property name exactly. A status_field value that does not match the casing used when entities were written will silently return zero matches, reading as HEALTHY.
• maxFiles caps the count this blueprint sees, not the true backlog size. If the real stuck-entity count exceeds max_entities_to_check, count reports the capped value and the task logs a warning (visible in the execution log) rather than the true total.
• This blueprint is UNTESTED against a real Azure Storage account or Azurite. The List task's filter/maxFiles properties and Output fields are taken from the plugin's own source on main, and the partition scalability figures are taken from Microsoft's own Table Storage documentation - but no Docker container or Kestra engine was run to execute this flow end to end.

Links

• Kestra Azure plugin documentation
• Azure Table Storage OData filter syntax documentation
• Azurite emulator documentation
• Kestra secrets (OSS vs Enterprise)