FileTransform

yaml

type: "io.kestra.plugin.scripts.groovy.FileTransform"

Transform ion format file from Kestra with a groovy script.

This allows you to transform the data, previously loaded by Kestra, as you need.

Take a ion format file from Kestra and iterate row per row. Each row will populate a row global variable. You need to alter this variable that will be saved on output file. If you set the row to null, the row will be skipped. You can create a variable rows to return multiple rows for a single row.

Examples

Convert row by row of a file from Kestra's internal storage.

yaml

id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "{{ outputs['avro-to-gcs'] }}"
script: |
  logger.info('row: {}', row)

  if (row.get('name') == 'richard') {
    row = null
  } else {
    row.put('email', row.get('name') + '@kestra.io')
  }

Create multiple rows from one row.

yaml

id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "{{ outputs['avro-to-gcs'] }}"
script: |
  logger.info('row: {}', row)
  rows = [["action", "insert"], row]

Transform a JSON string to a file.

yaml

id: "file_transform"
type: "io.kestra.plugin.scripts.groovy.FileTransform"
from: "[{\"name\":\"jane\"}, {\"name\":\"richard\"}]"
script: |
  logger.info('row: {}', row)

  if (row.get('name') == 'richard') {
    row = null
  } else {
    row.put('email', row.get('name') + '@kestra.io')
  }