Concat
yaml
type: "io.kestra.plugin.core.storage.Concat"
Concat files from the internal storage.
Examples
Concat 2 files with a custom separator.
yaml
id: "concat"
type: "io.kestra.plugin.core.storage.Concat"
files:
- "kestra://long/url/file1.txt"
- "kestra://long/url/file2.txt"
separator: "\n"
Concat files generated by an each task.
yaml
tasks:
- id: each
type: io.kestra.plugin.core.flow.ForEach
tasks:
- id: start_api_call
type: io.kestra.plugin.scripts.shell.Commands
commands:
- echo {{ taskrun.value }} > {{ temp.generated }}
files:
- generated
values: '["value1", "value2", "value3"]'
- id: concat
type: io.kestra.plugin.core.storage.Concat
files:
- "{{ outputs.start_api_call.value1.files.generated }}"
- "{{ outputs.start_api_call.value2.files.generated }}"
- "{{ outputs.start_api_call.value3.files.generated }}"
Concat a dynamic number of files.
yaml
tasks:
- id: echo
type: io.kestra.plugin.scripts.shell.Commands
commands:
- echo "Hello John" > {{ outputDirs.output }}/1.txt
- echo "Hello Jane" > {{ outputDirs.output }}/2.txt
- echo "Hello Doe" > {{ outputDirs.output }}/3.txt
outputDirs:
- output
- id: concat
type: io.kestra.plugin.core.storage.Concat
files: "{{ outputs.echo.files | jq('.[]') }}"
Properties
files
- Type: object
- Dynamic: ✔️
- Required: ✔️
List of files to be concatenated.
Must be a
kestra://
storage URIs, can be a list of string or json string
extension
- Type: string
- Dynamic: ✔️
- Required: ❌
- Default:
.tmp
The extension of the created file, default is .tmp.
separator
- Type: string
- Dynamic: ✔️
- Required: ❌
The separator to used between files, default is no separator.
Outputs
uri
- Type: string
- Required: ❌
- Format:
uri
The concatenated file URI.
Was this page helpful?