download
Download a file(s) from an S3 bucket.
Download a file(s) from an S3 bucket.
Download a file(s) from an S3 bucket.
This task can operate in two modes:
- Single file mode: when providing only the 'key' parameter, it downloads a specific file from S3.
- Multiple files mode: when using filtering parameters (prefix, delimiter, regexp), it downloads multiple files matching the criteria.
In single file mode, the output contains the properties of a single file (uri, contentLength, etc.). In multiple files mode, the output contains maps that associate each file key with its properties (uris, contentLengths, etc.).
type: "io.kestra.plugin.aws.s3.download"Examples
id: aws_s3_download
namespace: company.team
tasks:
- id: download
type: io.kestra.plugin.aws.s3.Download
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
bucket: "my-bucket"
key: "path/to/file"
Properties
bucket*Requiredstring
The S3 bucket name.
accessKeyIdstring
Access Key Id in order to connect to AWS.
compatibilityModebooleanstring
falseIf set to true, the task will use the AWS S3 DefaultAsyncClient instead of the S3CrtAsyncClient, which better integrates with S3-compatible services but restricts uploads and downloads to 2GB.
delimiterstring
A character used to group keys
When specified, the task switches to multiple files mode. The API returns all keys that share a common prefix up to the delimiter.
endpointOverridestring
The endpoint with which the SDK should communicate.
This property allows you to use a different S3 compatible storage backend.
expectedBucketOwnerstring
The account ID of the expected bucket owner
Requests will fail with a Forbidden error (access denied) if the bucket is owned by a different account.
forcePathStylebooleanstring
Force path style access.
Must only be used when compatibilityMode is enabled.
keystring
The key of a file to download
When specified without filtering options (prefix, delimiter, regexp), the task will download a single file.
markerstring
Used for pagination in multiple files mode
This is the key at which a previous listing ended.
maxKeysintegerstring
1000The maximum number of keys to include in the response in multiple files mode
prefixstring
The prefix of files to download
When specified, the task switches to multiple files mode and downloads all files with keys starting with this prefix.
regexpstring
A regular expression to filter the keys of the objects to download
When specified, the task switches to multiple files mode and only downloads files matching the pattern.
regionstring
AWS region with which the SDK should communicate.
requestPayerstring
Sets the value of the RequestPayer property for this object.
secretKeyIdstring
Secret Key Id in order to connect to AWS.
sessionTokenstring
AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource.
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
stsEndpointOverridestring
The AWS STS endpoint with which the SDKClient should communicate.
stsRoleArnstring
AWS STS Role.
The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
stsRoleExternalIdstring
AWS STS External Id.
A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.
stsRoleSessionDurationstring
PT15MdurationAWS STS Session duration.
The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.
stsRoleSessionNamestring
AWS STS Session name.
This property is only used when an stsRoleArn is defined.
versionIdstring
The specific version of the object
This property is only applicable when downloading a single file with the key parameter.
Outputs
contentLengthinteger
The size of the body in bytes
contentTypestring
A standard MIME type describing the format of the object data
filesobject
Map of object keys to their complete file information (multiple files mode only)
io.kestra.plugin.aws.s3.models.FileInfo
The size of the file in bytes
The MIME type of the file
An ETag is an opaque identifier assigned by a web server to a specific version of a resource found at a URL.
The metadata of the file
uriThe URI of the downloaded file in Kestra's storage
The version ID of the file
metadataobject
A map of metadata to store with the object in S3
uristring
uriversionIdstring
The version of the object.
Metrics
file.sizecounter
bytesThe size of the downloaded file.
