Launch Temporal Workflows in Batch per Entity from Kestra

Temporal excels at running one durable workflow per entity, a customer, an account, a device, but something has to decide each night which entities get one and start them all. This blueprint is that something: a ForEach iterates over a list of entity IDs and io.kestra.plugin.temporal.workflow.Trigger starts a CustomerRefresh workflow per entity on the customers task queue. Every workflow ID is templated from the entity ID, so rerunning the batch after a partial failure re-requests the same IDs instead of double-starting entities that already ran.

How it works

1. The entity_ids flow variable holds the demo batch inline; in production this list usually comes from an upstream query task.
2. launch_batch (io.kestra.plugin.core.flow.ForEach) iterates over the list; its direct child start_entity_workflow (io.kestra.plugin.temporal.workflow.Trigger) starts CustomerRefresh with the entity ID as a JSON-encoded argument and customer-refresh-{{ taskrun.value }} as the workflow ID. Deterministic IDs make Temporal the deduplication layer: an entity whose workflow is still running cannot be started twice.
3. Each start returns the documented workflowId and runId outputs, visible per task run in the execution.
4. notify posts the batch size, computed from the list length, to Slack once every start call has been accepted.
5. The errors block posts a distinct alert on any failed start, and a disabled-by-default Schedule trigger runs the batch nightly at 01:00.

What you get

• A nightly fan-out that turns a flat entity list into per-entity durable workflows.
• Rerun safety by construction: deterministic workflow IDs mean retries re-request rather than duplicate.
• Per-entity task runs in the execution history, so a single failing entity is identifiable at a glance.
• A batch-size confirmation in Slack and a partial-failure alert that never lets an incomplete batch look complete.

Who it's for

• Teams running per-customer or per-account processing as individual Temporal workflows.
• Data platform engineers triggering entity workflows after nightly upstream loads.
• Anyone replacing a cron-plus-script launcher with an auditable fan-out.

Why orchestrate this with Kestra

The per-entity business logic is exactly where it belongs, in Temporal workers with durable retries. What the workers cannot decide is the batch: which entities, when, and after which upstream data is ready. Kestra owns that layer, with schedules, upstream task chaining, secrets for the endpoint, and one execution per batch showing every start call it made.

Prerequisites

• A reachable Temporal frontend and a worker polling the customers task queue with a CustomerRefresh workflow registered.
• A Slack incoming webhook for batch reports and failure alerts.
• For Temporal Cloud, set the task's apiKey property; for mTLS clusters, provide caCert, clientCert, and clientKey.

Secrets

• TEMPORAL_ENDPOINT: Temporal frontend address in host:port format, for example temporal.example.com:7233.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the two secrets to your Kestra namespace.
2. Rename workflowType and taskQueue to match a workflow your workers register.
3. Execute the flow, confirm four workflows in the Temporal UI, then set disabled: false on the nightly trigger.

How to extend

• Replace the inline variable with an upstream database or API task and feed its output into the ForEach values.
• Add a date component to the workflow ID template when each night should start a fresh workflow per entity.
• Wait for each started workflow to complete by chaining the Wait task, as shown in the Temporal launch-and-wait blueprint.
• Cap concurrent starts with the ForEach task's concurrencyLimit property when the batch grows large.

Links

• Temporal plugin documentation
• Temporal documentation