GraalVM FileTransform

GraalVM FileTransform
Certified
Transform rows with Python on GraalVM

Streams rows from from (kestra:// URI, map, or list), lets Python mutate row, and writes the result as an ION file. Set concurrent to parallelize (order not preserved). Set row = None to drop a record; set rows array to emit multiple rows. Supports C-extension-backed stdlib modules such as ssl, sqlite3, and lzma, which requires enabling native access at the GraalVM engine level. Standard OS process APIs (os.system, subprocess) stay blocked, but this grant cannot be scoped down further: a script that reaches native code directly (e.g. ctypes) can still execute arbitrary OS commands or manipulate process memory. Unlike Kestra's Script/Shell tasks, which typically run in an isolated container or process via a TaskRunner, this code runs inline in the worker JVM process itself, so it has direct access to the worker process's own memory, file descriptors, and any secrets or other task state resident in that JVM. Only run scripts from users already trusted with the flow's credentials and infrastructure, as with any script task.

yaml
type: io.kestra.plugin.graalvm.python.FileTransform

Examples

yaml
id: transformPython
namespace: company.team

tasks:
  - id: download
    type: io.kestra.plugin.core.http.Download
    uri: https://dummyjson.com/carts/1
  - id: jsonToIon
    type: io.kestra.plugin.serdes.json.JsonToIon
    from: "{{outputs.download.uri}}"
  - id: transformPython
    type: io.kestra.plugin.graalvm.python.FileTransform
    from: "{{ outputs.jsonToIon.uri }}"
    script: |
      if row['id'] == 666:
        # remove un-needed row
        row = None
      else:
        # remove the 'products' column
        row['products'] = None
        # add a 'totalItems' column
        row['totalItems'] = row['totalProducts'] * row['totalQuantity']

Properties

from *string

Source file containing rows to transform

Accepts a kestra:// internal storage URI, map, or list; rows stream into the script before being rewritten as ION

script *string

Script body to execute

Template-rendered source code run by GraalVM in the selected language; flow variables are resolved before execution

concurrent integerstring

Parallel transformations to execute

Number of concurrent workers; ordering is not preserved when set and execution defaults to sequential when null

options object

Advanced GraalVM context options

Maps directly to GraalVM's Context.Builder#options: one entry per option key/value, for example python.WarnOptions or js.ecmascript-version. Keys that would weaken the sandbox already enforced by this task are rejected at execution time: python.PosixModuleBackend, any key related to host access or class loading, and any key prefixed with engine. or sandbox..