Storage Concat

Storage Concat

Certified

Concatenate files from Kestra internal storage.

Reads a list of kestra:// URIs (list or JSON string), streams them in order into a single file, and returns the new URI. Optional separator is inserted between files; output extension defaults to .tmp.

Use when you need to merge task outputs without downloading locally.

yaml
type: io.kestra.plugin.core.storage.Concat

Concat a 2 files.

yaml
    id: concat_example
    namespace: company.team

    tasks:
      - id: first_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello John" > name.txt
        outputFiles:
          - "name.txt"

      - id: second_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
        - echo "Hello Jane" > name.txt
        outputFiles:
          - "name.txt"

      - id: concat
        type: io.kestra.plugin.core.storage.Concat
        files:
          - "{{ outputs.first_file.outputFiles['name.txt'] }}"
          - "{{ outputs.second_file.outputFiles['name.txt'] }}"

Concat 2 files with a custom separator that adds a new line between each file.

yaml
    id: concat_example
    namespace: company.team

    tasks:
      - id: first_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello John
Goodbye John" > name.txt
        outputFiles:
          - "name.txt"

      - id: second_file
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello Jane
GoodbyeJane" > name.txt
        outputFiles:
          - "name.txt"

      - id: concat
        type: io.kestra.plugin.core.storage.Concat
        separator: "
"
        files:
          - "{{ outputs.first_file.outputFiles['name.txt'] }}"
          - "{{ outputs.second_file.outputFiles['name.txt'] }}"

Concat a dynamic number of files.

yaml
    id: concat_example
    namespace: company.team

    tasks:
      - id: generate_files
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo "Hello John" > 1.txt
          - echo "Hello Jane" > 2.txt
          - echo "Hello Doe" > 3.txt
        outputFiles:
          - "*.txt"

      - id: concat
        type: io.kestra.plugin.core.storage.Concat
        files: "{{ outputs.generate_files.outputFiles | jq('.[]') }}"

Concat files generated by a Loop task.

yaml
id: concat_example
namespace: company.team

tasks:
  - id: loop
    type: io.kestra.plugin.core.flow.Loop
    values: ["value1", "value2", "value3"]
    outputs:
      - id: file
        type: STRING
        value: "{{outputs.start_api_call.outputFiles['data.txt']}}"
    tasks:
      - id: start_api_call
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo {{ item.value }} > data.txt
        outputFiles:
          - data.txt
  - id: concat_foreach_manual
    type: io.kestra.plugin.core.storage.Concat
    files: "{{ outputs.loop.outputs | jq('.[].outputs.file') }}"
Properties

List of files to be concatenated

Must be kestra:// storage URIs; it can be a list of strings or a JSON string.

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
Default.tmp

The extension of the created file — the default is .tmp

The separator to used between files — the default is no separator

Formaturi

The concatenated file URI

More Plugins in this Category

Docker
Docker

Docker tasks for building images, running containers, and managing artifacts from Kestra workflows, plus managing AI models through the Docker Model Runner.

CoreInfrastructureAI

14 tasks

23 blueprints

Deno
Deno

Tasks that run Deno scripts.

DataCore

4 tasks