Downloads
Download multiple files from a GCS bucket.
Download multiple files from a GCS bucket.
Download multiple files from a GCS bucket.
type: "io.kestra.plugin.gcp.gcs.Downloads"Examples
Download a list of files and move it to an archive folders
id: gcp_gcs_downloads
namespace: company.team
tasks:
- id: downloads
type: io.kestra.plugin.gcp.gcs.Downloads
from: gs://my-bucket/kestra/files/
action: MOVE
moveDirectory: gs://my-bucket/kestra/archive/
Properties
action*Requiredstring
MOVEDELETENONEThe action to perform on the retrieved files. If using 'NONE' make sure to handle the files inside your flow to avoid infinite triggering.
from*Requiredstring
The directory to list
allVersionsbooleanstring
If set to true, lists all versions of a blob. The default is false.
impersonatedServiceAccountstring
The GCP service account to impersonate.
listingTypestring
DIRECTORYRECURSIVEDIRECTORYThe listing type you want (like directory or recursive)
if DIRECTORY, will only list objects in the specified directory
if RECURSIVE, will list objects in the specified directory recursively
Default value is DIRECTORY
When using RECURSIVE value, be careful to move your files to a location not in the from scope
moveDirectorystring
The destination directory for MOVE action.
projectIdstring
The GCP project ID.
regExpstring
A regexp to filter on full path
ex:
regExp: .* to match all files
regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv
scopesarray
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used.
serviceAccountstring
The GCP service account.
Outputs
blobsarray
The bucket of the downloaded file
io.kestra.plugin.gcp.gcs.models.Blob
date-timedate-timedate-timedate-timedate-timeurioutputFilesobject
The downloaded files as a map of from/to URIs.
