AWS Consume

AWS Consume

Certified

Consume messages from SQS

Polls a queue until maxRecords or maxDuration is reached, stores messages to internal storage, and optionally auto-deletes them.

yaml
type: io.kestra.plugin.aws.sqs.Consume
yaml
id: aws_sqs_consume
namespace: company.team

tasks:
  - id: consume
    type: io.kestra.plugin.aws.sqs.Consume
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    queueUrl: "https://sqs.eu-central-1.amazonaws.com/000000000000/test-queue"
Properties

The SQS queue URL. The queue must already exist

Access Key Id in order to connect to AWS

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

Defaulttrue

Auto-delete

If true (default), delete messages after processing.

Enable compatibility mode

Use it to connect to S3 bucket with S3 compatible services that don't support the new transport client.

DefaultPT5S

The amount of time to wait when acquiring a connection from the pool before giving up and timing out

The endpoint with which the SDK should communicate

This property allows you to use a different S3 compatible storage backend.

Force path style access

Must only be used when compatibilityMode is enabled.

Default50

Maximum number of allowed concurrent requests

For HTTP/1.1 this is the same as max connections. For HTTP/2 the number of connections that will be used depends on the max streams allowed per connection. If the maximum number of concurrent requests is exceeded they may be queued in the HTTP client (see maxPendingConnectionAcquires) and can cause increased latencies. If the client is overloaded enough such that the pending connection queue fills up, subsequent requests may be rejected or time out (see connectionAcquisitionTimeout).

Max duration

Stop after this duration elapses.

Max records

Stop after consuming this many messages.

Reference (ref) of the pluginDefaults to apply to this task.

AWS region with which the SDK should communicate

Secret Key Id in order to connect to AWS

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

DefaultSTRING
Possible Values
STRINGJSON

The serializer/deserializer to use

AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

The AWS STS endpoint with which the SDKClient should communicate

AWS STS Role

The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

AWS STS External Id

A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.

DefaultPT15M

AWS STS Session duration

The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.

AWS STS Session name

This property is only used when an stsRoleArn is defined.

Default30

Visibility timeout

Seconds a received message stays hidden; default 30s.

Consumed count

Messages read within limits.

Formaturi

Messages file URI

Internal storage URI with serialized messages.

Unitmessages

Number of messages consumed from the SQS queue.

More Plugins in this Category

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks

PostgreSQL
PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Data

6 tasks

25 blueprints