FileTransform
FileTransform
yaml
type: "io.kestra.plugin.scripts.groovy.FileTransform"
Transform ion format file from Kestra's internal storage with a Groovy script.
Examples
Convert row by row of a file from Kestra's internal storage.
yaml
id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "{{ outputs['avro-to-gcs'] }}"
script: |
logger.info('row: {}', row)
// remove a column
row.remove('useless_column')
// update a column
row['email'] = row['name'] + '@kestra.io'
// set a column to null
row['last_update'] = null
Create multiple rows from one row.
yaml
id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "{{ outputs['avro-to-gcs'] }}"
script: |
logger.info('row: {}', row)
rows = [["action", "insert"], row]
Transform a JSON string to a file.
yaml
id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "[{\"name\":\"jane\"}, {\"name\":\"richard\"}]"
script: |
logger.info('row: {}', row)
if (row.get('name') == 'richard') {
row = null
} else {
row.put('email', row.get('name') + '@kestra.io')
}
Properties
from
- Type: string
- Dynamic: ✔️
- Required: ✔️
Source file containing rows to transform. Can be Kestra's internal storage URI, a map or a list.
concurrent
- Type: integer
- Dynamic: ❌
- Required: ❌
- Minimum:
>= 2
Number of concurrent parallel transformations to execute. Take care that the order is not respected if you use parallelism.
script
- Type: string
- Dynamic: ✔️
- Required: ❌
A full script.
Outputs
uri
- Type: string
- Dynamic: ❓
- Required: ❌
- Format:
uri
URI of a temporary result file. The file will be serialized as ion file.