Core Plugins and tasks UploadFiles

Core Plugins and tasks UploadFiles
Certified
Upload files into a Namespace.

Sends files to Namespace storage via glob patterns (files) or explicit maps (filesMap). destination controls the target folder (leading slash required); conflict sets overwrite/skip/error behavior.

Accepts WorkingDirectory outputs or other task file maps so you can rename or relocate files while uploading.

yaml
type: io.kestra.plugin.core.namespace.UploadFiles

Examples

Upload files generated by a previous task using the filesMap property.

yaml
id: upload_files_from_git
namespace: company.team

tasks:
  - id: download
    type: io.kestra.plugin.core.http.Download
    uri: https://github.com/kestra-io/scripts/archive/refs/heads/main.zip

  - id: unzip
    type: io.kestra.plugin.compress.ArchiveDecompress
    from: "{{ outputs.download.uri }}"
    algorithm: ZIP

  - id: upload
    type: io.kestra.plugin.core.namespace.UploadFiles
    filesMap: "{{ outputs.unzip.files }}"
    namespace: "{{ flow.namespace }}"

Upload a folder using a glob pattern. Note that the Regex syntax requires a glob pattern inspired by Apache Ant patterns. Make sure that your pattern starts with glob: , followed by the pattern. For example, use glob: **/dbt/** to upload the entire dbt folder (with all files and subdirectories) regardless of that folder's location in the directory structure.

yaml
id: upload_dbt_project
namespace: company.team

tasks:
  - id: wdir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: git_clone
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: master

      - id: upload
        type: io.kestra.plugin.core.namespace.UploadFiles
        files:
          - "glob:**/dbt/**"
        namespace: "{{ flow.namespace }}"

Upload a specific file and rename it.

yaml
id: upload_a_file
namespace: company.team

tasks:
  - id: download
    type: io.kestra.plugin.core.http.Download
    uri: https://github.com/kestra-io/scripts/archive/refs/heads/main.zip

  - id: unzip
    type: io.kestra.plugin.compress.ArchiveDecompress
    from: "{{ outputs.download.uri }}"
    algorithm: ZIP

  - id: upload
    type: io.kestra.plugin.core.namespace.UploadFiles
    filesMap:
      LICENCE: "{{ outputs.unzip.files['scripts-main/LICENSE'] }}"
    namespace: "{{ flow.namespace }}"

Properties

namespace *string

The namespace to which the files will be uploaded

conflict string

DefaultOVERWRITE

Possible Values

OVERWRITEERRORSKIP

Which action to take when uploading a file that already exists

Can be one of the following options: OVERWRITE, ERROR or SKIP. Default is OVERWRITE.

destination string

Default/

The destination folder

Required when providing a list of files.

files array

SubTypestring

A list of Regex that match files in the current directory

This should be a list of Regex matching the Apache Ant patterns.It's primarily intended to be used with the WorkingDirectory task

filesMap objectstring

A map of key-value pairs where the key is the filename and the value is the URI of the file to upload.

This should be a map of URI, with the key being the filename that will be upload and the key the URI.This property is intended to be used with the output files of other tasks. Many Kestra tasks, incl. all Downloads tasks, output a map of files so that you can directly pass the output property to this task e.g. outputFiles in the S3 Downloads task or the files in the Archive Decompress task.