Consume messages periodically from Azure Event Hubs and create one execution per batch.
If you would like to consume each message from Azure Event Hubs in real-time and create one execution per message, you can use the io.kestra.plugin.azure.eventhubs.RealtimeTrigger instead.
type: "io.kestra.plugin.azure.eventhubs.Trigger"Trigger flow based on events received from Azure Event Hubs in batch.
id: azure_eventhubs_trigger
namespace: company.team
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: Hello there! I received {{ trigger.eventsCount }} from Azure EventHubs!
triggers:
- id: read_from_eventhub
type: io.kestra.plugin.azure.eventhubs.Trigger
interval: PT30S
eventHubName: my_eventhub
namespace: my_eventhub_namespace
connectionString: "{{ secret('EVENTHUBS_CONNECTION') }}"
bodyDeserializer: JSON
consumerGroup: "$Default"
checkpointStoreProperties:
containerName: kestra
connectionString: "{{ secret('BLOB_CONNECTION') }}"
The event hub to read from.
Namespace name of the event hub to connect to.
The Deserializer to be used for serializing the event value.
The config properties to be passed to the Deserializer.
Configs in key/value pairs.
The config properties to be used for configuring the BlobCheckpointStore.
Azure Event Hubs Checkpoint Store can be used for storing checkpoints while processing events from Azure Event Hubs.
The maximum number of retry attempts before considering a client operation to have failed.
The maximum permissible delay between retry attempts in milliseconds.
Connection string of the Storage Account.
The consumer group.
Custom endpoint address when connecting to the Event Hubs service.
The ISO Datetime to be used when PartitionStartingPosition is configured to INSTANT.
Configs in key/value pairs.
Interval between polling.
The interval between 2 different polls of schedule, this can avoid to overload the remote system with too many calls. For most of the triggers that depend on external systems, a minimal interval must be at least PT30S. See ISO_8601 Durations for more information of available interval values.
The maximum number of events to consume per event hub partition per poll.
The max time duration to wait to receive events from all partitions.
The max time duration to wait to receive a batch of events up to the maxBatchSizePerPartition.
The starting position.
The SAS token to use for authenticating requests.
This string should only be the query parameters (with or without a leading '?') and not a full URL.
List of execution states after which a trigger should be stopped (a.k.a. disabled).
Number of events consumed from Azure Event Hubs.
URI of a kestra internal storage file containing the messages.