Core Plugins and tasks Concat

Core Plugins and tasks Concat

Certified

Concatenate files from Kestra internal storage.

Reads a list of kestra:// URIs (list or JSON string), streams them in order into a single file, and returns the new URI. Optional separator is inserted between files; output extension defaults to .tmp.

Use when you need to merge task outputs without downloading locally.

yaml
type: io.kestra.plugin.core.storage.Concat

Concat a 2 files.

yaml
    id: concat_example
    namespace: company.team

    tasks:
      - id: first_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello John" > name.txt
        outputFiles:
          - "name.txt"

      - id: second_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
        - echo "Hello Jane" > name.txt
        outputFiles:
          - "name.txt"

      - id: concat
        type: io.kestra.plugin.core.storage.Concat
        files:
          - "{{ outputs.first_file.outputFiles['name.txt'] }}"
          - "{{ outputs.second_file.outputFiles['name.txt'] }}"

Concat 2 files with a custom separator that adds a new line between each file.

yaml
    id: concat_example
    namespace: company.team

    tasks:
      - id: first_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello John
Goodbye John" > name.txt
        outputFiles:
          - "name.txt"

      - id: second_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello Jane
GoodbyeJane" > name.txt
        outputFiles:
          - "name.txt"

      - id: concat
        type: io.kestra.plugin.core.storage.Concat
        separator: "
"
        files:
          - "{{ outputs.first_file.outputFiles['name.txt'] }}"
          - "{{ outputs.second_file.outputFiles['name.txt'] }}"

Concat a dynamic number of files.

yaml
    id: concat_example
    namespace: company.team

    tasks:
      - id: generate_files
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello John" > 1.txt
          - echo "Hello Jane" > 2.txt
          - echo "Hello Doe" > 3.txt
        outputFiles:
          - "*.txt"

      - id: concat
        type: io.kestra.plugin.core.storage.Concat
        files: "{{ outputs.generate_files.outputFiles | jq('.[]') }}"

Concat files generated by a ForEach task.

yaml
id: concat_example
namespace: company.team

tasks:
  - id: foreach
    type: io.kestra.plugin.core.flow.ForEach
    values: ["value1", "value2", "value3"]
    tasks:
      - id: start_api_call
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo {{ taskrun.value }} > data.txt
        outputFiles:
          - data.txt

  - id: concat_foreach_manual
    type: io.kestra.plugin.core.storage.Concat
    files: "{{ outputs.start_api_call | jq('.[].outputFiles.generated') }}"
Properties

List of files to be concatenated

Must be kestra:// storage URIs; it can be a list of strings or a JSON string.

Default.tmp

The extension of the created file — the default is .tmp

Reference (ref) of the pluginDefaults to apply to this task.

The separator to used between files — the default is no separator

Formaturi

The concatenated file URI

More Plugins in this Category

Compression
Compression

Tasks that compress and decompress files and archives for Kestra flows.

Core

6 tasks

4 blueprints

Cryptography
Cryptography

Tasks that secure files in Kestra pipelines using OpenPGP encryption, decryption, and signature validation.

CoreInfrastructure

2 tasks