Download

Download a file(s) from an S3 bucket.

This task can operate in two modes:

Single file mode: when providing only the 'key' parameter, it downloads a specific file from S3.
Multiple files mode: when using filtering parameters (prefix, delimiter, regexp), it downloads multiple files matching the criteria.

In single file mode, the output contains the properties of a single file (uri, contentLength, etc.). In multiple files mode, the output contains maps that associate each file key with its properties (uris, contentLengths, etc.).

yaml
type: "io.kestra.plugin.aws.s3.Download"

Examples

yaml
id: aws_s3_download
namespace: company.team

tasks:
  - id: download
    type: io.kestra.plugin.aws.s3.Download
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    key: "path/to/file"

Properties

bucket *string

The S3 bucket name.

accessKeyId string

Access Key Id in order to connect to AWS.

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

compatibilityMode booleanstring

Default false

If set to true, the task will use the AWS S3 DefaultAsyncClient instead of the S3CrtAsyncClient, which better integrates with S3-compatible services but restricts uploads and downloads to 2GB.

delimiter string

A character used to group keys

When specified, the task switches to multiple files mode. The API returns all keys that share a common prefix up to the delimiter.

endpointOverride string

The endpoint with which the SDK should communicate.

This property allows you to use a different S3 compatible storage backend.

expectedBucketOwner string

The account ID of the expected bucket owner

Requests will fail with a Forbidden error (access denied) if the bucket is owned by a different account.

forcePathStyle booleanstring

Force path style access.

Must only be used when compatibilityMode is enabled.

key string

The key of a file to download

When specified without filtering options (prefix, delimiter, regexp), the task will download a single file.

marker string

Used for pagination in multiple files mode

This is the key at which a previous listing ended.

maxKeys integerstring

Default 1000

The maximum number of keys to include in the response in multiple files mode

prefix string

The prefix of files to download

When specified, the task switches to multiple files mode and downloads all files with keys starting with this prefix.

regexp string

A regular expression to filter the keys of the objects to download

When specified, the task switches to multiple files mode and only downloads files matching the pattern.

region string

AWS region with which the SDK should communicate.

requestPayer string

Sets the value of the RequestPayer property for this object.

secretKeyId string

Secret Key Id in order to connect to AWS.

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

sessionToken string

AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource.

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

stsEndpointOverride string

The AWS STS endpoint with which the SDKClient should communicate.

stsRoleArn string

AWS STS Role.

The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

stsRoleExternalId string

AWS STS External Id.

A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.

stsRoleSessionDuration string

Default PT15M

Format duration

AWS STS Session duration.

The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.

stsRoleSessionName string

AWS STS Session name.

This property is only used when an stsRoleArn is defined.

versionId string

The specific version of the object

This property is only applicable when downloading a single file with the key parameter.

Outputs

contentLength integer

The size of the body in bytes

contentType string

A standard MIME type describing the format of the object data

files object

SubType

Map of object keys to their complete file information (multiple files mode only)

metadata object

SubType string

A map of metadata to store with the object in S3

uri string

Format uri

versionId string

The version of the object.

Definitions

io.kestra.plugin.aws.s3.models.FileInfo

contentLength integer

The size of the file in bytes

contentType string

The MIME type of the file

eTag string

An ETag is an opaque identifier assigned by a web server to a specific version of a resource found at a URL.

metadata object

SubType string

The metadata of the file

uri string

Format uri

The URI of the downloaded file in Kestra's storage

versionId string

The version ID of the file

​Download

Download