Compose

List file on a GCS bucket.

yaml
type: "io.kestra.plugin.gcp.gcs.Compose"

Concat files in a bucket

yaml
id: gcp_gcs_compose
namespace: company.team

tasks:
  - id: compose
    type: io.kestra.plugin.gcp.gcs.Compose
    list:
      from: "gs://my_bucket/dir/"
    to: "gs://my_bucket/destination/my-compose-file.txt"

Dynamic YES

The directory to list

Dynamic YES

Default false

Dynamic YES

The GCP service account to impersonate.

Dynamic YES

The GCP project ID.

SubType string

Dynamic YES

Default ["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used.

Dynamic YES

The GCP service account.

Dynamic YES

The destination path

Format uri

Dynamic YES

The directory to list

Dynamic YES

Default BOTH

Possible Values

FILESDIRECTORYBOTH

Dynamic YES

Default DIRECTORY

Possible Values

RECURSIVEDIRECTORY

The listing type you want (like directory or recursive)

if DIRECTORY, will only list objects in the specified directory if RECURSIVE, will list objects in the specified directory recursively Default value is DIRECTORY When using RECURSIVE value, be careful to move your files to a location not in the from scope

Dynamic YES

A regexp to filter on full path

ex: regExp: .* to match all files regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv

​Compose

Compose