Datagen RealtimeTrigger
CertifiedStream generated data in real time
Datagen RealtimeTrigger
Stream generated data in real time
Continuously runs the generator with backpressure. Controls include throughput per second (minimum 1), maxRecords cap (default unlimited), and reportingInterval for stats; data is emitted in trigger output, not stored.
type: io.kestra.plugin.datagen.core.RealtimeTriggerExamples
id: datagen_person_realtime_trigger_json
namespace: com.example.datagen
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "Created: {{ trigger.value.name }} ({{ trigger.value.email }})!"
triggers:
- id: datagen
type: io.kestra.plugin.datagen.core.RealtimeTrigger
throughput: 10
reportingInterval: PT5S
generator:
type: io.kestra.plugin.datagen.generators.JsonObjectGenerator
locale: ["fr", "FR"]
value:
name: "#{name.fullName}"
email: "#{internet.emailAddress}"
age: 30
address:
city: "#{address.city}"
zip: "#{address.zipCode}"
skills: [ "#{job.keySkills}", "#{job.position}", "hardcoded" ]
ts: "{{ now() }}"
Properties
generator *RequiredNon-dynamic
Data generator
Generator used for each emitted record; not stored, only emitted in trigger outputs.
Generate JSON objects from templates
Renders the value map, then evaluates Datafaker expressions (#{...}) on every record. Supports nested maps/lists; locale list overrides Faker locale, otherwise the library default is used.
Object template
Map of key-value pairs rendered per record; strings starting with #{ are evaluated by Datafaker, including nested maps and lists.
Locales
Optional locale list in the format [language, country, variant]; empty list uses Faker's default locale.
Generate random byte arrays
Allocates a byte array of the configured size and fills it with SecureRandom bytes. Size should be positive; randomness relies on the JVM's default seed.
0Byte array size
Number of bytes produced per record; should be greater than zero
Generate strings from templates
Renders the value string (Pebble) and then evaluates Datafaker expressions like #{name.firstName} for each record. Locale list overrides Faker locale; empty list uses the library default.
String template
String rendered per record; supports Pebble variables and Datafaker expressions starting with #{
Locales
Optional locale list in the format [language, country, variant]; empty list uses Faker's default locale.
allowConcurrent Non-dynamicboolean
falseSpecifies whether a trigger is allowed to start a new execution even if a previous run is still in progress.
maxRecords integerstring
9223372036854775807Total number of records
Hard limit on records emitted; defaults to unlimited when not set.
reportingInterval string
PT15SReporting interval
Period for throughput reporting; defaults to 15 seconds.
stopAfter Non-dynamicarray
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTEDList of execution states after which a trigger should be stopped (a.k.a. disabled).
throughput integerstring
1Trigger throughput
Approximate records per second; values below 1 are rounded up to 1.
when string
trueA condition that determines whether the trigger should run.
A Pebble expression evaluated at trigger time. The trigger fires only when the expression evaluates to a truthy value (true, a non-empty string, a non-zero number). Use this to gate trigger execution on dynamic runtime values such as execution labels, flow variables, or environment conditions.
Outputs
count integer
Items generated
Number of records produced by the generator.
size integer
Generated size (bytes)
Total size in bytes of the returned value or stored ION file.
uri string
uriStored file URI
URI in internal storage when store is true; null for inline outputs.
value object
Generated value
Inline content when store is false; null when data is stored. May be string, number, JSON object, or byte array depending on the generator.
