LocalFiles

yaml

type: "io.kestra.core.tasks.storages.LocalFiles"

Allow to create files in the local filesystem or to send files from the local filesystem to the Kestra's internal storage.

This task should be used with the WorkingDirectory task to be able to access the same local filesystem within multiple tasks. Note that this task cannot be skipped, so setting disabled: true will not work on this task.

Examples

Output local files created in a Python task and load them to S3.

yaml

    id: outputsFromPythonTask
    namespace: dev

    tasks:
      - id: wdir
        type: io.kestra.core.tasks.flows.WorkingDirectory
        tasks:
          - id: cloneRepository
            type: io.kestra.plugin.git.Clone
            url: https://github.com/kestra-io/examples
            branch: main

          - id: gitPythonScripts
            type: io.kestra.plugin.scripts.python.Commands
            warningOnStdErr: false
            runner: DOCKER
            docker:
              image: ghcr.io/kestra-io/pydata:latest
            beforeCommands:
              - pip install faker > /dev/null
            commands:
              - python scripts/etl_script.py
              - python scripts/generate_orders.py

          - id: exportFiles
            type: io.kestra.core.tasks.storages.LocalFiles
            outputs:
              - orders.csv
              - "*.parquet"

      - id: loadCsvToS3
        type: io.kestra.plugin.aws.s3.Upload
        accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
        secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
        region: eu-central-1
        bucket: kestraio
        key: stage/orders.csv
        from: "{{ outputs.exportFiles.outputFiles['orders.csv'] }}"
        disabled: true

Create a local file that will be accessible to a bash task.

yaml

id: "local-files"
namespace: "io.kestra.tests"

tasks:
  - id: workingDir
    type: io.kestra.core.tasks.flows.WorkingDirectory
    tasks:
    - id: inputFiles
      type: io.kestra.core.tasks.storages.LocalFiles
      inputs:
        hello.txt: "Hello World\n"
        address.json: "{{ outputs.myTaskId.uri }}"
    - id: bash
      type: io.kestra.plugin.scripts.shell.Commands
      commands:
        - cat hello.txt

Send local files to Kestra's internal storage.

yaml

id: "local-files"
namespace: "io.kestra.tests"

tasks:
  - id: workingDir
    type: io.kestra.core.tasks.flows.WorkingDirectory
    tasks:
    - id: bash
      type: io.kestra.plugin.scripts.shell.Commands
      commands:
        - mkdir -p sub/dir
        - echo "Hello from Bash" >> sub/dir/bash1.txt
        - echo "Hello from Bash" >> sub/dir/bash2.txt
    - id: outputFiles
      type: io.kestra.core.tasks.storages.LocalFiles
      outputs:
        - sub/**