Commands icon
Process icon
Trigger icon

React to an SQS trigger

Build an event-driven pipeline that polls an Amazon SQS queue and runs a Kestra flow on every new message, with retries, lineage, and declarative YAML.

Categories
Cloud
id: react-to-sqs-trigger
namespace: company.team

tasks:
  - id: print_message
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - cat "{{ trigger.uri }}"

triggers:
  - id: sqs
    type: io.kestra.plugin.aws.sqs.Trigger
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
    region: "{{ secret('AWS_DEFAULT_REGION') }}"
    queueUrl: https://sqs.eu-central-1.amazonaws.com/123456789/kestra
    maxRecords: 1

Turn Amazon SQS into an event source for your data and automation pipelines. This blueprint polls an existing SQS queue and launches a Kestra flow execution every time a new message arrives, so downstream processing starts the moment work is queued instead of waiting on a fixed schedule. It is a practical pattern for fan-out processing, decoupled microservices, and reacting to events emitted by other AWS services through SQS.

How it works

  • The io.kestra.plugin.aws.sqs.Trigger continuously polls the queue at queueUrl using AWS credentials supplied via accessKeyId, secretKeyId, and region.
  • maxRecords is set to 1, so the flow fires once per message and processes them one at a time.
  • When a message is consumed, Kestra writes its content to internal storage and exposes it as {{ trigger.uri }}.
  • The print_message task (io.kestra.plugin.scripts.shell.Commands) runs cat "{{ trigger.uri }}" on the io.kestra.plugin.core.runner.Process runner to read and print that message payload.

What you get

  • A working event-driven trigger that reacts to SQS messages in near real time.
  • Per-message executions you can inspect, replay, and audit individually.
  • A clear template for plugging real business logic in place of the demo cat task.

Who it's for

  • Data engineers wiring AWS event sources into orchestrated pipelines.
  • Platform teams decoupling producers and consumers with queues.
  • Developers who want message-driven workflows without managing a custom consumer service.

Why orchestrate this with Kestra

SQS delivers messages but does not run your pipeline, handle retries on downstream steps, or give you visibility into what happened after a message was consumed. Kestra fills that gap: the event trigger starts a fully observable execution per message, tasks can declare retries and error handling, every run is captured for lineage and replay, and the whole pipeline stays as version-controlled declarative YAML.

Prerequisites

  • An existing Amazon SQS queue and its queue URL.
  • An AWS IAM identity with permission to receive and delete messages on that queue.

Secrets

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_DEFAULT_REGION

Quick start

  1. Store AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION as Kestra secrets.
  2. Replace the queueUrl value with the URL of your own SQS queue.
  3. Add the flow and enable the trigger.
  4. Send a test message to the queue and watch a new execution appear.

How to extend

  • Swap the print_message task for tasks that parse, validate, and route the payload.
  • Increase maxRecords to batch multiple messages per execution.
  • Branch on message content to load data into a warehouse, call an API, or notify a team.
  • Add retry and error tasks to make downstream processing resilient.

Links

Orchestrate with Kestra
Orchestrate AWS with Kestra
Share this Blueprint
See How

New to Kestra?

Use blueprints to kickstart your first workflows.