RealtimeTrigger
Trigger flows from Azure Event Hubs in real time
RealtimeTrigger
Trigger flows from Azure Event Hubs in real time
yaml
type: io.kestra.plugin.azure.eventhubs.RealtimeTriggerExamples
yaml
id: azure_eventhubs_realtime_trigger
namespace: company.team
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: Hello there! I received {{ trigger.body }} from Azure EventHubs!
triggers:
- id: read_from_eventhub
type: io.kestra.plugin.azure.eventhubs.RealtimeTrigger
eventHubName: my_eventhub
namespace: my_eventhub_namespace
connectionString: "{{ secret('EVENTHUBS_CONNECTION') }}"
bodyDeserializer: JSON
consumerGroup: "$Default"
checkpointStoreProperties:
containerName: kestra
connectionString: "{{ secret('BLOB_CONNECTION') }}"
yaml
id: eventhubs_realtime_trigger
namespace: company.team
tasks:
- id: insert_into_storagetable
type: io.kestra.plugin.azure.storage.table.Bulk
endpoint: https://yourstorageaccount.blob.core.windows.net
connectionString: "{{ secret('STORAGETABLE_CONNECTION') }}"
table: orders
from:
- partitionKey: order_id
rowKey: "{{ trigger.body | jq('.order_id') | first }}"
properties:
customer_name: "{{ trigger.body | jq('.customer_name') | first }}"
customer_email: "{{ trigger.body | jq('.customer_email') | first }}"
product_id: "{{ trigger.body | jq('.product_id') | first }}"
price: "{{ trigger.body | jq('.price') | first }}"
quantity: "{{ trigger.body | jq('.quantity') | first }}"
total: "{{ trigger.body | jq('.total') | first }}"
triggers:
- id: realtime_trigger
type: io.kestra.plugin.azure.eventhubs.RealtimeTrigger
eventHubName: orders
namespace: kestra
connectionString: "{{ secret('EVENTHUBS_CONNECTION') }}"
bodyDeserializer: JSON
consumerGroup: $Default
checkpointStoreProperties:
containerName: kestra
connectionString: "{{ secret('BLOB_CONNECTION') }}"
Properties
eventHubName *Requiredstring
namespace *Requiredstring
allowConcurrent Non-dynamicboolean
Default
falsebodyDeserializer string
Default
STRINGPossible Values
STRINGBINARYIONJSONbodyDeserializerProperties object
Default
{}checkpointStoreProperties object
SubTypestring
Default
{}clientMaxRetries integerstring
Default
5clientRetryDelay integerstring
Default
500conditions Non-dynamic
Definitions
Allow events only between two datetimes.
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.DateTimeBetweenio.kestra.core.models.conditions.types.DateTimeBetweenConditionio.kestra.plugin.core.condition.DateTimeBetweenConditionafterstring
beforestring
datestring
Default
{{ trigger.date }}Allow events on a specific weekday.
dayOfWeek*Requiredstring
Possible Values
MONDAYTUESDAYWEDNESDAYTHURSDAYFRIDAYSATURDAYSUNDAYtype*Requiredobject
Possible Values
io.kestra.plugin.core.condition.DayWeekio.kestra.core.models.conditions.types.DayWeekConditionio.kestra.plugin.core.condition.DayWeekConditiondatestring
Default
{{ trigger.date }}Allow events on an nth weekday within the month.
dayInMonth*Requiredstring
Possible Values
FIRSTLASTSECONDTHIRDFOURTHdayOfWeek*Requiredstring
Possible Values
MONDAYTUESDAYWEDNESDAYTHURSDAYFRIDAYSATURDAYSUNDAYtype*Requiredobject
Possible Values
io.kestra.plugin.core.condition.DayWeekInMonthio.kestra.plugin.core.condition.DayWeekInMonthConditionio.kestra.core.models.conditions.types.DayWeekInMonthConditiondatestring
Default
{{ trigger.date }}Match events from a specific flow.
flowId*Requiredstring
namespace*Requiredstring
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.ExecutionFlowio.kestra.plugin.core.condition.ExecutionFlowConditionio.kestra.core.models.conditions.types.ExecutionFlowConditionMatch executions by label set.
labels*Requiredarrayobject
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.ExecutionLabelsio.kestra.core.models.conditions.types.ExecutionLabelsConditionio.kestra.plugin.core.condition.ExecutionLabelsConditionMatch executions by namespace.
namespace*Requiredstring
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.ExecutionNamespaceio.kestra.core.models.conditions.types.ExecutionNamespaceConditionio.kestra.plugin.core.condition.ExecutionNamespaceConditioncomparisonstring
Possible Values
EQUALSPREFIXSUFFIXprefixbooleanstring
Default
falseCondition based on the outputs of an upstream execution.
expression*Requiredbooleanstring
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.ExecutionOutputsio.kestra.core.models.conditions.types.ExecutionOutputsConditionio.kestra.plugin.core.condition.ExecutionOutputsConditionMatch executions by status.
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.ExecutionStatusio.kestra.core.models.conditions.types.ExecutionStatusConditionio.kestra.plugin.core.condition.ExecutionStatusConditioninarray
SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTEDnotInarray
SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTEDCondition based on variable expression.
expression*Requiredstring
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.Expressionio.kestra.plugin.core.condition.ExpressionConditionio.kestra.core.models.conditions.types.VariableConditionMatch executions where a task was retried.
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.HasRetryAttemptio.kestra.plugin.core.condition.HasRetryAttemptConditionio.kestra.core.models.conditions.types.HasRetryAttemptConditioninarray
SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTEDnotInarray
SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTEDInvert one or more conditions.
conditions*Required
Min items
1type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.Notio.kestra.core.models.conditions.types.NotConditionio.kestra.plugin.core.condition.NotConditionPass when any condition is true.
conditions*Required
Min items
1type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.Orio.kestra.core.models.conditions.types.OrConditionio.kestra.plugin.core.condition.OrConditionAllow events on public holidays.
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.PublicHolidayio.kestra.plugin.core.condition.PublicHolidayConditionio.kestra.core.models.conditions.types.PublicHolidayConditioncountrystring
datestring
Default
{{ trigger.date}}subDivisionstring
Allow events between two times of day.
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.TimeBetweenio.kestra.plugin.core.condition.TimeBetweenConditionio.kestra.core.models.conditions.types.TimeBetweenConditionafterstring
beforestring
datestring
Default
{{ trigger.date }}Allow events on weekends.
type*Requiredobject
Possible Values
io.kestra.plugin.core.condition.Weekendio.kestra.plugin.core.condition.WeekendConditionio.kestra.core.models.conditions.types.WeekendConditiondatestring
Default
{{ trigger.date }}connectionString string
consumerGroup string
Default
$DefaultcustomEndpointAddress string
enqueueTime string
partitionStartingPosition string
Default
EARLIESTPossible Values
EARLIESTLATESTINSTANTsasToken string
stopAfter Non-dynamicarray
SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED