Blueprints

Download a file and upload it to S3

Source

yaml
id: upload-file-to-s3
namespace: company.team

inputs:
  - id: bucket
    type: STRING
    defaults: declarative-data-orchestration

  - id: file_url
    type: STRING
    defaults: https://wri-dataportal-prod.s3.amazonaws.com/manual/global_power_plant_database_v_1_3.zip

tasks:
  - id: download_file
    type: io.kestra.plugin.core.http.Download
    uri: "{{ inputs.file_url }}"

  - id: upload_to_s3
    type: io.kestra.plugin.aws.s3.Upload
    from: "{{ outputs.download_file.uri }}"
    key: powerplant/global_power_plant_database.zip
    bucket: "{{ inputs.bucket }}"
    region: "{{ secret('AWS_DEFAULT_REGION') }}"
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
See more

About this blueprint

S3 Inputs

This flow downloads a single file and uploads it to an S3 bucket. This flow assumes AWS credentials stored as secrets AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION.

Download

Upload

More Related Blueprints

S3 Parallel Variables Inputs Outputs

List objects in an S3 bucket and process them in parallel

S3 Parallel Inputs Software Engineering Outputs

Download a zip file, unzip it and upload all files in parallel to AWS S3 - using pluginDefaults to avoid boilerplate code

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra