Concat Concat

yaml
type: "io.kestra.plugin.core.storage.Concat"

Concat files from the internal storage.

Examples

Concat 2 files with a custom separator.

yaml
id: "concat"
type: "io.kestra.plugin.core.storage.Concat"
files: 
  - "kestra://long/url/file1.txt"
  - "kestra://long/url/file2.txt"
separator: "\n"

Concat files generated by an each task.

yaml
tasks:
  - id: each
    type: io.kestra.plugin.core.flow.ForEach
    tasks:
      - id: start_api_call
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo {{ taskrun.value }} > {{ temp.generated }}
        files:
          - generated
    values: '["value1", "value2", "value3"]'

  - id: concat
    type: io.kestra.plugin.core.storage.Concat
    files:
      - "{{ outputs.start_api_call.value1.files.generated }}"
      - "{{ outputs.start_api_call.value2.files.generated }}"
      - "{{ outputs.start_api_call.value3.files.generated }}"

Concat a dynamic number of files.

yaml
tasks:
  - id: echo
    type: io.kestra.plugin.scripts.shell.Commands
    commands:
      - echo "Hello John" > {{ outputDirs.output }}/1.txt
      - echo "Hello Jane" > {{ outputDirs.output }}/2.txt
      - echo "Hello Doe" > {{ outputDirs.output }}/3.txt
    outputDirs:
      - output
  - id: concat
    type: io.kestra.plugin.core.storage.Concat
    files: "{{ outputs.echo.files | jq('.[]') }}"

Properties

files

  • Type: object
  • Dynamic: ✔️
  • Required: ✔️

List of files to be concatenated.

Must be a kestra:// storage URIs, can be a list of string or json string

extension

  • Type: string
  • Dynamic: ✔️
  • Required:
  • Default: .tmp

The extension of the created file, default is .tmp.

separator

  • Type: string
  • Dynamic: ✔️
  • Required:

The separator to used between files, default is no separator.

Outputs

uri

  • Type: string
  • Required:
  • Format: uri

The concatenated file URI.

Was this page helpful?