🚀 New! Kestra raises $3 million to grow Learn more

pluginsplugin-gcptasksgcsio.kestra.plugin.gcp.gcs.downloads

​Downloads

Downloads Downloads

yaml
type: "io.kestra.plugin.gcp.gcs.Downloads"

Download multiple files from a GCS bucket.

Examples

Download a list of files and move it to an archive folders

yaml
id: "downloads"
type: "io.kestra.plugin.gcp.gcs.Downloads"
from: gs://my-bucket/kestra/files/
action: MOVE
moveDirectory: gs://my-bucket/kestra/archive/

Properties

action

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️
  • Possible Values:
    • MOVE
    • DELETE

The action to do on find files

from

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The directory to list

allVersions

  • Type: boolean
  • Dynamic: ✔️
  • Required: ❌

If set to true, lists all versions of a blob. The default is false.

listingType

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: DIRECTORY
  • Possible Values:
    • RECURSIVE
    • DIRECTORY

The listing type you want (like directory or recursive)

if DIRECTORY, will only list objects in the specified directory if RECURSIVE, will list objects in the specified directory recursively Default value is DIRECTORY When using RECURSIVE value, be carefull to move your files to a location not in the from scope

moveDirectory

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The destination directory in case off MOVE

projectId

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP project id

regExp

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

A regexp to filter on full path

ex: regExp: .* to match all files regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv

scopes

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: [https://www.googleapis.com/auth/cloud-platform]

The GCP scopes to used

serviceAccount

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The GCP service account key

Outputs

blobs

  • Type: array
  • SubType: Blob

The bucket of the downloaded file

Definitions

Blob

bucket

  • Type: string
  • Dynamic: ❓
  • Required: ❌

cacheControl

  • Type: string
  • Dynamic: ❓
  • Required: ❌

componentCount

  • Type: integer
  • Dynamic: ❓
  • Required: ❌

contentDisposition

  • Type: string
  • Dynamic: ❓
  • Required: ❌

contentEncoding

  • Type: string
  • Dynamic: ❓
  • Required: ❌

contentLanguage

  • Type: string
  • Dynamic: ❓
  • Required: ❌

contentType

  • Type: string
  • Dynamic: ❓
  • Required: ❌

crc32c

  • Type: string
  • Dynamic: ❓
  • Required: ❌

createTime

  • Type: string
  • Dynamic: ❓
  • Required: ❌

customTime

  • Type: string
  • Dynamic: ❓
  • Required: ❌

deleteTime

  • Type: string
  • Dynamic: ❓
  • Required: ❌

etag

  • Type: string
  • Dynamic: ❓
  • Required: ❌

eventBasedHold

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌

generatedId

  • Type: string
  • Dynamic: ❓
  • Required: ❌

isDirectory

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌

kmsKeyName

  • Type: string
  • Dynamic: ❓
  • Required: ❌

md5

  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Type: string
  • Dynamic: ❓
  • Required: ❌

metaGeneration

  • Type: integer
  • Dynamic: ❓
  • Required: ❌

metadata

  • Type: object
  • SubType: string
  • Dynamic: ❓
  • Required: ❌

name

  • Type: string
  • Dynamic: ❓
  • Required: ❌

retentionExpirationTime

  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Type: string
  • Dynamic: ❓
  • Required: ❌

size

  • Type: integer
  • Dynamic: ❓
  • Required: ❌

temporaryHold

  • Type: boolean
  • Dynamic: ❓
  • Required: ❌

timeStorageClassUpdated

  • Type: string
  • Dynamic: ❓
  • Required: ❌

updateTime

  • Type: string
  • Dynamic: ❓
  • Required: ❌

uri

  • Type: string
  • Dynamic: ❓
  • Required: ❌