ESC
Ask Kestra AI
ESC
Kestra AI

Hi! I'm your Kestra AI assistant.
Ask me anything about workflows.

EXAMPLE QUESTIONS

What is a task runner?
How to handle errors & retry on flow?
How to run Python script?

Alibaba Cloud Downloads

Alibaba Cloud Downloads

Certified

Download several objects from an OSS bucket

Lists the objects of a bucket by prefix and regular expression, downloads them into Kestra internal storage, and can then leave, delete or move each of them. Objects are downloaded first and only then deleted or moved, so nothing is removed from the bucket before it is safely in internal storage. An object that cannot be downloaded is skipped and left in place; the task only fails when none of the matching objects can be downloaded. If the delete or move of an object fails, the task keeps going and flags that object with actionFailed.

yaml
type: io.kestra.plugin.alibaba.oss.Downloads

Download the CSV files of a prefix and move them to an archive prefix

yaml
id: oss_downloads
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.alibaba.oss.Downloads
    accessKeyId: "{{ secret('ALIBABA_ACCESS_KEY_ID') }}"
    accessKeySecret: "{{ secret('ALIBABA_ACCESS_KEY_SECRET') }}"
    region: cn-hangzhou
    bucket: my-bucket
    prefix: landing/
    regexp: ".*\\.csv"
    action: MOVE
    moveToPrefix: archive/
Properties

AccessKey ID

The AccessKey ID used to authenticate against Alibaba Cloud.

AccessKey secret

The AccessKey secret paired with the AccessKey ID.

Bucket

The name of the OSS bucket.

DefaultNONE
Possible Values
NONEDELETEMOVE

Action

What to do with each object once it is downloaded: NONE leaves it, DELETE removes it, MOVE copies it under moveToPrefix and removes the original.

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150

Endpoint override

Custom http or https endpoint URL. When set, it takes precedence over the endpoint derived from region.

Default25

Maximum number of files

Maximum number of objects downloaded, between 1 and 1000 (default 25). A warning is logged when more objects match.

Move destination bucket

The bucket objects are moved to when action is MOVE. Defaults to bucket.

Move destination prefix

Prefix prepended to the original key when action is MOVE. Required for MOVE. The task fails when a moved key in the same bucket would still match prefix and regexp, because such objects would be matched again by the next run.

Defaultfalse

Path-style access

Address buckets as <endpoint>/<bucket> instead of <bucket>.<endpoint>. Useful with custom endpoints such as a local emulator.

Prefix

Only objects whose key starts with this prefix are downloaded.

Regular expression

Only objects whose full key matches this Java regular expression are downloaded. It is limited to 256 characters and evaluation is time-boxed to guard against catastrophic backtracking.

Region

The Alibaba Cloud region ID, for example cn-hangzhou. The service endpoint is derived from it unless endpointOverride is set.

Security token

Optional Security Token Service (STS) token, required when using temporary credentials.

The downloaded objects

Definitions
actionFailedboolean

Whether the delete or move failed

When true, the object is still in the bucket.

etagstring

ETag

keystring

Object key

lastModifiedstring
Formatdate-time

Last modified time

sizeinteger

Size in bytes

uristring
Formaturi

URI of the downloaded object in Kestra internal storage

More Plugins in this Category

Ashby
Ashby

Ashby plugin for Kestra

Data

1 tasks

1 blueprints

SAP HANA
SAP HANA

Tasks that query SAP HANA using the official SAP HANA JDBC driver (ngdbc.jar).

Data

3 tasks

3 blueprints