AWS List

AWS List

Certified

List objects in an S3 bucket

Lists objects with optional prefix/regex filtering and caps returned items with maxFiles. Emits object count metric.

yaml
type: io.kestra.plugin.aws.s3.List
yaml
id: aws_s3_list
namespace: company.team

tasks:
  - id: list
    type: io.kestra.plugin.aws.s3.List
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "sub-dir"
Properties

The S3 bucket where to download the file

Access Key Id in order to connect to AWS

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

Enable compatibility mode

Use it to connect to S3 bucket with S3 compatible services that don't support the new transport client.

A delimiter is a character you use to group keys

The EncodingType property for this object

The endpoint with which the SDK should communicate

This property allows you to use a different S3 compatible storage backend.

The account ID of the expected bucket owner

If the bucket is owned by a different account, the request fails with the HTTP status code 403 Forbidden (access denied).

DefaultBOTH
Possible Values
FILESDIRECTORYBOTH

The type of objects to filter: files, directory, or both

Force path style access

Must only be used when compatibilityMode is enabled.

Marker is where you want Amazon S3 to start listing from

Amazon S3 starts listing after this specified key. Marker can be any key in the bucket.

Default25

Max files

Truncates results to this count; default 25.

Default1000

Maximum number of objects returned across all paginated S3 calls

Total upper bound on the objects returned. It also sets the per-page size sent to S3, which AWS caps at 1000. The listing pages until this total is reached or the bucket is exhausted. Default 1000.

Reference (ref) of the pluginDefaults to apply to this task.

Limits the response to keys that begin with the specified prefix

A regexp to filter on full key

ex: regExp: .* to match all files regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv

AWS region with which the SDK should communicate

Sets the value of the RequestPayer property for this object

Secret Key Id in order to connect to AWS

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

The AWS STS endpoint with which the SDKClient should communicate

AWS STS Role

The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

AWS STS External Id

A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.

DefaultPT15M

AWS STS Session duration

The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.

AWS STS Session name

This property is only used when an stsRoleArn is defined.

Objects

Listed objects after filtering and maxFiles limit.

Definitions
checksumAlgorithmstring
checksumValuestring
etagstring
keystring
lastModifiedstring
Formatdate-time
owner
displayNamestring
idstring
sizeinteger
uristring
Formaturi
Unitobjects

The number of objects returned by the S3 list operation.

More Plugins in this Category

NetBox (EE)
NetBox
Enterprise

Enterprise NetBox tasks to create, update, and query devices, IP addresses, and sites over the NetBox API.

Infrastructure

7 tasks

4 blueprints

Clickhouse
Clickhouse

Tasks that connect to ClickHouse via JDBC to run queries and loads.

Data

5 tasks

2 blueprints