Google Cloud Downloads
CertifiedDownload multiple GCS objects
Google Cloud Downloads
Download multiple GCS objects
Lists objects under a prefix (with optional regex and versioning) then downloads them to Kestra storage. Can MOVE (copy then delete) or DELETE source objects after download.
type: io.kestra.plugin.gcp.gcs.DownloadsExamples
Download a list of files and move it to an archive folders
id: gcp_gcs_downloads
namespace: company.team
tasks:
- id: downloads
type: io.kestra.plugin.gcp.gcs.Downloads
from: gs://my-bucket/kestra/files/
action: MOVE
moveDirectory: gs://my-bucket/kestra/archive/
Download a list of files with end-to-end checksum validation
id: gcp_gcs_downloads_validated
namespace: company.team
tasks:
- id: downloads
type: io.kestra.plugin.gcp.gcs.Downloads
from: gs://my-bucket/kestra/files/
action: DELETE
validateChecksum: true
Properties
from *string
Source prefix
gs:// bucket path to list and download from
action string
NONEMOVEDELETENONEPost-download action
NONE (default), DELETE, or MOVE (copy to moveDirectory then delete source)
allVersions booleanstring
If set to true, lists all versions of a blob. The default is false
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150impersonatedServiceAccount string
The GCP service account to impersonate
listingType string
DIRECTORYRECURSIVEDIRECTORYListing type
DIRECTORY lists only immediate children; RECURSIVE traverses all descendants. Default DIRECTORY.
maxFiles integerstring
25Max files
Maximum number of files to list and download
moveDirectory string
Move destination
Required when action is MOVE; gs:// prefix for moved objects
projectId string
The GCP project ID
regExp string
Regex filter
Optional regex applied to object names
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serviceAccount string
The GCP service account
validateChecksum booleanstring
falseValidate each downloaded file against the GCS checksum
When true, every downloaded file is streamed through a CRC32C hasher (or MD5 if the object has no CRC32C) and compared to the checksum reported by Google Cloud Storage. If any file mismatches, the task fails. Used for end-to-end integrity, not for security/authentication.
Outputs
blobs array
Downloaded objects
io.kestra.plugin.gcp.gcs.models.Blob
date-timedate-timedate-timedate-timedate-timeurioutputFiles object
Downloaded file map
Map of object names to Kestra storage URIs
Metrics
checksum.validated counter
Number of files whose checksum was successfully validated after download.
