Download data and upload to Google Cloud Storage

Source

yaml

id: load-to-cloud-storage
namespace: company.team

tasks:
  - id: data
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv

  - id: cloud_storage
    type: io.kestra.plugin.gcp.gcs.Upload
    from: "{{ outputs.data.uri }}"
    to: gs://kestra-demo/data.csv

About this blueprint

Software Engineering GCP

This blueprint shows how to download a CSV file via http Download, and upload it to GCS.

Note: Authentication to GCP can be done by setting the GOOGLE_APPLICATION_CREDENTIALS variable in environment (via a service account for example).

Download

Upload

More Related Blueprints

Software Engineering GCP

Read a file from inputs and upload it to GCS

Docker Python Software Engineering GCP

Run Python script in a Docker container based on Google Artifact Registry container image

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra