FileTransform FileTransform

yaml
type: "io.kestra.plugin.scripts.groovy.FileTransform"

Transform ion format file from Kestra's internal storage with a Groovy script.

Examples

Convert row by row of a file from Kestra's internal storage.

yaml
id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "{{ outputs['avro-to-gcs'] }}"
script: |
  logger.info('row: {}', row)

  // remove a column
  row.remove('useless_column')
  // update a column
  row['email'] = row['name'] + '@kestra.io'
  // set a column to null
  row['last_update'] = null

Create multiple rows from one row.

yaml
id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "{{ outputs['avro-to-gcs'] }}"
script: |
  logger.info('row: {}', row)
  rows = [["action", "insert"], row]

Transform a JSON string to a file.

yaml
id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "[{\"name\":\"jane\"}, {\"name\":\"richard\"}]"
script: |
  logger.info('row: {}', row)

  if (row.get('name') == 'richard') {
    row = null
  } else {
    row.put('email', row.get('name') + '@kestra.io')
  }

Properties

from

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

Source file containing rows to transform. Can be Kestra's internal storage URI, a map or a list.

concurrent

  • Type: integer
  • Dynamic:
  • Required:
  • Minimum: >= 2

Number of concurrent parallel transformations to execute. Take care that the order is not respected if you use parallelism.

script

  • Type: string
  • Dynamic: ✔️
  • Required:

A full script.

Outputs

uri

  • Type: string
  • Dynamic:
  • Required:
  • Format: uri

URI of a temporary result file. The file will be serialized as ion file.