Consume RabbitMQ orders with dead letter queue handling

Give every RabbitMQ order message a traceable, retryable home instead of an opaque consumer process. This blueprint streams messages from an orders.queue queue one execution at a time, validates each payload, forwards valid orders to a downstream API, and republishes anything that fails, whether from bad data or a failed API call, to a dead letter exchange it provisions on its own. No message is ever silently dropped or left stuck at the head of the queue.

How it works

1. io.kestra.plugin.amqp.RealtimeTrigger (on_new_order) opens a manual-ack subscription on the orders.queue queue with serdeType: JSON, launching one execution the moment a message arrives and exposing its deserialized body as trigger.data.
2. declare_dead_letter_exchange (io.kestra.plugin.amqp.DeclareExchange) idempotently declares a FANOUT exchange named orders.dlx.
3. create_dead_letter_queue (io.kestra.plugin.amqp.CreateQueue) idempotently creates the durable orders.dlq queue.
4. bind_dead_letter_queue (io.kestra.plugin.amqp.QueueBind) binds orders.dlq to orders.dlx, so anything published to the exchange lands in the queue. All three provisioning calls are safe to repeat on every execution, since RabbitMQ treats a declare or bind of something that already exists as a no-op.
5. log_order (io.kestra.plugin.core.log.Log) records the raw message for traceability.
6. assert_valid_order (io.kestra.plugin.core.execution.Fail) checks the deserialized payload for order_id and amount; a message missing either field fails the execution immediately instead of reaching the downstream API.
7. process_order (io.kestra.plugin.core.http.Request) POSTs the order to the downstream order processing API; a non-2xx response fails the task the same way an invalid message does.
8. A flow-level errors block runs publish_to_dead_letter (io.kestra.plugin.amqp.Publish), which republishes the original trigger.data to orders.dlx with a failedExecutionId header, whether the failure came from validation or from the API call.

What you get

• One Kestra execution per RabbitMQ message, with full run history, logs, and retries for every order instead of an opaque consumer process.
• A self-provisioning dead letter exchange and queue that a fresh environment can bootstrap without a separate setup script.
• A single validation gate that rejects malformed messages before they reach the downstream API, plus one errors block that dead letters both validation failures and API failures.
• A failedExecutionId header on every dead-lettered message, so a person or a follow-up flow can look up exactly which execution rejected it.

Who it's for

• Backend and platform teams running RabbitMQ who need poison-message handling without writing a custom consumer.
• Data engineers bridging event queues into REST APIs or databases who want per-message observability.
• Teams migrating off ad hoc consumer scripts (pika, amqplib) that have no retry, replay, or dead-lettering built in.

Why orchestrate this with Kestra

RabbitMQ can enforce dead-lettering natively with an x-dead-letter-exchange policy, but that keeps failure handling implicit and invisible outside the broker. Kestra's errors block makes the same pattern explicit and observable: every rejected message is tied to a specific execution, with logs, the exact validation or HTTP failure, and a full replay history. The RealtimeTrigger also removes the need to write and operate a long-running consumer process at all, since Kestra manages the subscription, retries, and concurrency for you.

Prerequisites

• A running RabbitMQ broker with an existing orders.queue queue that upstream producers publish to.
• A downstream order processing API that accepts a JSON POST body.

Secrets

• RABBITMQ_HOST, RABBITMQ_PORT, RABBITMQ_USERNAME, RABBITMQ_PASSWORD, RABBITMQ_VHOST: broker connection details.
• ORDERS_API_URL: endpoint of the downstream order processing API.

Quick start

1. Add the secrets above to your Kestra namespace.
2. Confirm orders.queue exists and something publishes order messages to it (see the produce-to-rabbitmq blueprint for a producer pattern).
3. Add this flow. On its first execution it creates orders.dlx and orders.dlq and binds them together.
4. Publish a message missing amount to orders.queue and confirm it appears in orders.dlq instead of failing silently.

How to extend

• Move the three provisioning tasks into a separate one-time setup flow if you would rather not repeat idempotent declare calls on every message.
• Add a retry policy to process_order so a transient API failure does not immediately dead letter a valid order.
• Swap the FANOUT exchange for a TOPIC exchange with a routing key to split dead letters by failure type.
• Chain a scheduled flow that periodically drains orders.dlq with io.kestra.plugin.amqp.Consume and reprocesses or archives the failed messages.

Links

• Kestra AMQP plugin: https://kestra.io/plugins/plugin-amqp