MinIO Downloads

MinIO Downloads

Certified

Downloads multiple files from a MinIO bucket

Downloads multiple objects matching a prefix or pattern from a MinIO bucket to Kestra's internal storage.

yaml
type: io.kestra.plugin.minio.Downloads
yaml
id: minio_downloads
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.minio.Downloads
    accessKeyId: "<access-key>"
    secretKeyId: "{{ secret('MINIO_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "sub-dir"
    action: "DELETE"

Download files from an S3-compatible storage — here, Spaces Object Storage from Digital Ocean.

yaml
id: s3_compatible_downloads
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.minio.Downloads
    accessKeyId: "<access-key>"
    secretKeyId: "{{ secret('MINIO_SECRET_KEY_ID') }}"
    endpoint: https://<region>.digitaloceanspaces.com
    bucket: "kestra-test-bucket"
    prefix: "data/orders"
    action: "DELETE"
Properties
Possible Values
MOVEDELETENONE

The action to perform on the retrieved files. If using 'NONE' make sure to handle the files inside your flow to avoid infinite triggering

Access Key Id for authentication

The bucket name

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections to custom MinIO endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to MinIO (mTLS).

A delimiter is a character you use to group keys

URL to the MinIO endpoint

DefaultBOTH
Possible Values
FILESDIRECTORYBOTH

The type of objects to filter: files, directory, or both

Marker is where you want to start listing from

Start listing after this specified key. Marker can be any key in the bucket.

Default1000

Sets the maximum number of keys returned in the response

By default, the action returns up to 1,000 key names. The response might contain fewer keys but will never contain more.

The destination bucket and key for MOVE action

Definitions
bucketstring

The bucket name

keystring

The bucket key

Reference (ref) of the pluginDefaults to apply to this task.

Limits the response to keys that begin with the specified prefix

A regexp to filter on full key

ex: regExp: .* to match all files regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv

MinIO region with which the SDK should communicate

Secret Key Id for authentication

SSL/TLS configuration options

Definitions
insecureTrustAllCertificatesbooleanstring

Whether to disable checking of the remote SSL certificate.

Only applies if no trust store is configured. Note: This makes the SSL connection insecure and should only be used for testing. If you are using a self-signed certificate, set up a trust store instead.

The list of objects

Definitions
etagstring
keystring
lastModifiedstring
Formatdate-time
owner
displayNamestring
idstring
sizeinteger
uristring
Formaturi
SubTypestring

The downloaded files as a map of from/to URIs

Unitbytes

The size in bytes of the downloaded file from the MinIO bucket.

More Plugins in this Category

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks

PostgreSQL
PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Data

6 tasks

25 blueprints