Google Cloud Downloads

Google Cloud Downloads

Certified

Download multiple GCS objects

Lists objects under a prefix (with optional regex and versioning) then downloads them to Kestra storage. Can MOVE (copy then delete) or DELETE source objects after download.

yaml
type: io.kestra.plugin.gcp.gcs.Downloads

Download a list of files and move it to an archive folders

yaml
id: gcp_gcs_downloads
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.gcp.gcs.Downloads
    from: gs://my-bucket/kestra/files/
    action: MOVE
    moveDirectory: gs://my-bucket/kestra/archive/

Download a list of files with end-to-end checksum validation

yaml
id: gcp_gcs_downloads_validated
namespace: company.team

tasks:
  - id: downloads
    type: io.kestra.plugin.gcp.gcs.Downloads
    from: gs://my-bucket/kestra/files/
    action: DELETE
    validateChecksum: true
Properties

Source prefix

gs:// bucket path to list and download from

DefaultNONE
Possible Values
MOVEDELETENONE

Post-download action

NONE (default), DELETE, or MOVE (copy to moveDirectory then delete source)

If set to true, lists all versions of a blob. The default is false

The GCP service account to impersonate

DefaultDIRECTORY
Possible Values
RECURSIVEDIRECTORY

Listing type

DIRECTORY lists only immediate children; RECURSIVE traverses all descendants. Default DIRECTORY.

Default25

Max files

Maximum number of files to list and download

Move destination

Required when action is MOVE; gs:// prefix for moved objects

Reference (ref) of the pluginDefaults to apply to this task.

The GCP project ID

Regex filter

Optional regex applied to object names

SubTypestring
Default["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used

The GCP service account

Defaultfalse

Validate each downloaded file against the GCS checksum

When true, every downloaded file is streamed through a CRC32C hasher (or MD5 if the object has no CRC32C) and compared to the checksum reported by Google Cloud Storage. If any file mismatches, the task fails. Used for end-to-end integrity, not for security/authentication.

Downloaded objects

Definitions
bucketstring
cacheControlstring
componentCountinteger
contentDispositionstring
contentEncodingstring
contentLanguagestring
contentTypestring
crc32cstring
createTimestring
Formatdate-time
customTimestring
Formatdate-time
deleteTimestring
Formatdate-time
etagstring
eventBasedHoldboolean
generatedIdstring
isDirectoryboolean
kmsKeyNamestring
md5string
mediaLinkstring
metaGenerationinteger
metadataobject
SubTypestring
namestring
retentionExpirationTimeinteger
selfLinkstring
sizeinteger
temporaryHoldboolean
timeStorageClassUpdatedstring
Formatdate-time
updateTimestring
Formatdate-time
uristring
Formaturi
SubTypestring

Downloaded file map

Map of object names to Kestra storage URIs

Number of files whose checksum was successfully validated after download.

More Plugins in this Category

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks

PostgreSQL
PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Data

6 tasks

25 blueprints