Google Cloud Storage (GCS) Upload

Google Cloud Storage (GCS) Upload

Certified

Upload a file to GCS

Reads a file from Kestra internal storage and writes it to a gs:// destination. Supports content metadata settings.

yaml
type: io.kestra.plugin.gcp.gcs.Upload

Upload a FILE input to GCS

yaml
id: gcp_gcs_upload
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: upload
    type: io.kestra.plugin.gcp.gcs.Upload
    from: "{{ inputs.file }}"
    to: "gs://my_bucket/dir/file.csv"

Download data and upload to Google Cloud Storage

yaml
id: load_to_cloud_storage
namespace: company.team

tasks:
  - id: data
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv

  - id: cloud_storage
    type: io.kestra.plugin.gcp.gcs.Upload
    from: "{{ outputs.data.uri }}"
    to: gs://kestra-demo/data.csv
Installation

Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:

kestractl plugins get io.kestra.plugin:plugin-gcp --compatible-for $KESTRA_VERSION

For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.

Properties

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150

Cache-Control

Content-Disposition

Content-Encoding

Content-Type

Expected CRC32C of the source

Optional base64-encoded big-endian CRC32C. When set, the task fails if the source data read from Kestra internal storage does not match this checksum.

Expected MD5 of the source

Optional base64-encoded MD5 digest. When set, the task fails if the source data read from Kestra internal storage does not match this digest. Use this to assert end-to-end integrity against a checksum produced upstream.

Source file URI

Kestra internal storage URI to upload

The GCP service account to impersonate

The GCP project ID

SubTypestring
Default["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used

The GCP service account

When unset or rendered to an empty value, Application Default Credentials are used (e.g. GOOGLE_APPLICATION_CREDENTIALS or the runtime's attached identity).

Destination object URI

gs:// bucket/path for the uploaded object. Special characters and partition brackets are automatically percent-encoded if unencoded.

Defaulttrue

Validate checksum after upload

When true (default), compute MD5 and CRC32C client-side while streaming and compare them against what GCS reports server-side once the upload completes. When false, the implicit transit integrity check is skipped; explicit expectedMd5/expectedCrc32c assertions are still honored when set.

Audience of the Workload Identity Federation provider

The full resource name of the workload identity pool provider, //iam.googleapis.com/projects/<number>/locations/global/workloadIdentityPools/<pool>/providers/<provider>. Required with workloadIdentityToken, and must match the aud claim of the token.

OIDC token for Workload Identity Federation

When set together with workloadIdentityAudience, the token is exchanged at Google STS for federated credentials and no service account key is needed: Google verifies it against the identity provider that issued it. Use {{ oidcToken(audience='…') }} to have Kestra Enterprise mint one for the running execution. Combine with impersonatedServiceAccount to act as a service account the pool may impersonate, and set projectId explicitly since no key file carries it. Cannot be combined with serviceAccount.

Base64-encoded big-endian CRC32C of the uploaded data

Base64-encoded MD5 of the uploaded data

Formaturi

Uploaded object URI

Unitbytes

Size of the uploaded file.