Datagen Generate

Datagen Generate

Certified

Generate synthetic data

Runs the configured generator (e.g., Datafaker) once (inline) or for a batch when store is true. When stored, results are written as Ion lines to internal storage; defaults are store=false and batchSize=1.

yaml
type: io.kestra.plugin.datagen.core.Generate
yaml
id: datagen_person_json
namespace: com.example.datagen

tasks:
  - id: datagen
    type: io.kestra.plugin.datagen.core.Generate
    generator:
      type: io.kestra.plugin.datagen.generators.JsonObjectGenerator
      locale: ["fr", "FR"]
      value:
        name: "#{name.fullName}"
        email: "#{internet.emailAddress}"
        age: 30
        address:
          city: "#{address.city}"
          zip: "#{address.zipCode}"
        skills: [ "#{job.keySkills}", "#{job.position}", "hardcoded" ]
        ts: "{{ now() }}"

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "Created: {{ outputs.datagen.value.name }} ({{ outputs.datagen.value.email }})!"
yaml
id: datagen_person_json_batch
namespace: com.example.datagen

tasks:
  - id: datagen
    type: io.kestra.plugin.datagen.core.Generate
    batchSize: 100
    store: true
    generator:
      type: io.kestra.plugin.datagen.generators.JsonObjectGenerator
      locale: ["fr", "FR"]
      value:
        name: "#{name.fullName}"
        email: "#{internet.emailAddress}"
        age: 30
        address:
          city: "#{address.city}"
          zip: "#{address.zipCode}"
        skills: [ "#{job.keySkills}", "#{job.position}", "hardcoded" ]
        ts: "{{ now() }}"

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "Created: {{ outputs.datagen.value.name }} ({{ outputs.datagen.value.email }})!"
yaml
id: datagen_person_csv
namespace: com.example.datagen
inputs:
  - id: rows
    type: INT
    displayName: "Number of Rows"
    description: "Number of data rows to generate (excluding header)."
    defaults: 10

  - id: separator
    type: STRING
    displayName: "Separator"
    description: "Column separator character (e.g., ',', ';')."
    defaults: ","

  - id: header
    type: BOOL
    displayName: "Include header"
    description: "Include header row in the CSV output."
    defaults: true
tasks:
  - id: datagen
    type: io.kestra.plugin.datagen.core.Generate
    generator:
      type: io.kestra.plugin.datagen.generators.StringValueGenerator
      locale: ["en", "US"]
      value: |
        #{csv '{{ inputs.separator }}','"','{{inputs.header}}','{{ inputs.rows}}','name_column','#{Name.first_name}','last_name_column','#{Name.last_name}'}

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: |
        {{ outputs.datagen.value }}
Properties

Choose data generator

Generator used for each record; required for all generate tasks and triggers.

Definitions

Renders the value map, then evaluates Datafaker expressions (#{...}) on every record. Supports nested maps/lists; locale list overrides Faker locale, otherwise the library default is used.

type*Requiredobject
value*Requiredobject

Object template

Map of key-value pairs rendered per record; strings starting with #{ are evaluated by Datafaker, including nested maps and lists.

localearray
SubTypestring

Locales

Optional locale list in the format [language, country, variant]; empty list uses Faker's default locale.

Allocates a byte array of the configured size and fills it with SecureRandom bytes. Size should be positive; randomness relies on the JVM's default seed.

type*Requiredobject
sizeinteger
Default0

Byte array size

Number of bytes produced per record; should be greater than zero

Renders the value string (Pebble) and then evaluates Datafaker expressions like #{name.firstName} for each record. Locale list overrides Faker locale; empty list uses the library default.

type*Requiredobject
value*Requiredstring

String template

String rendered per record; supports Pebble variables and Datafaker expressions starting with #{

localearray
SubTypestring

Locales

Optional locale list in the format [language, country, variant]; empty list uses Faker's default locale.

Default1

Batch size

Number of items to generate when storing data; only used if store is true. Defaults to 1.

Reference (ref) of the pluginDefaults to apply to this task.

Defaultfalse

Store generated data

Persist output to Kestra internal storage as Ion lines when true; defaults to false to return the value inline.

Items generated

Number of records produced by the generator.

Generated size (bytes)

Total size in bytes of the returned value or stored Ion file.

Formaturi

Stored file URI

URI in internal storage when store is true; null for inline outputs.

Generated value

Inline content when store is false; null when data is stored. May be string, number, JSON object, or byte array depending on the generator.

More Plugins in this Category

Clickhouse
Clickhouse

Tasks that connect to ClickHouse via JDBC to run queries and loads.

Data

5 tasks

2 blueprints

Weaviate
Weaviate

Tasks that manage and query vectors in Weaviate.

DataAI

4 tasks

2 blueprints