Azure Consume

Azure Consume

Certified

Consume events from Azure Event Hubs

Polls Event Hubs partitions in batches, checkpoints to Azure Blob Storage, and writes events to internal storage as Ion. Defaults: consumerGroup=$Default, partitionStartingPosition=EARLIEST, maxBatchSizePerPartition=50, maxWaitTimePerPartition=PT5S, maxDuration=PT10S. Requires checkpointStoreProperties.connectionString and .containerName.

yaml
type: io.kestra.plugin.azure.eventhubs.Consume

Consume data events from Azure EventHubs.

yaml
id: azure_eventhubs_consume_data_events
namespace: company.team

tasks:
  - id: consume_from_eventhub
    type: io.kestra.plugin.azure.eventhubs.Consume
    eventHubName: my_eventhub
    namespace: my_eventhub_namespace
    connectionString: "{{ secret('EVENTHUBS_CONNECTION') }}"
    bodyDeserializer: JSON
    consumerGroup: "$Default"
    checkpointStoreProperties:
      containerName: kestra
      connectionString: "{{ secret('BLOB_CONNECTION') }}"
Properties

Checkpoint store properties

Blob container config for checkpoints (connectionString, containerName required)

Event Hub name

Target Event Hub entity

Namespace

Event Hubs namespace when using AAD or shared key auth

DefaultSTRING
Possible Values
STRINGBINARYIONJSON

Body deserializer

Serde used to decode event bodies; defaults to STRING

Default{}

Deserializer properties

Key/value options passed to the selected serde

Default5

Client max retries

Max retry attempts for Event Hubs client operations; default 5

Default500

Retry delay (ms)

Delay between client retries in milliseconds; default 500

Event Hubs connection string

Namespace or Event Hub–level connection string; overrides key/sas fields when set

Default$Default

Consumer group

Event Hubs consumer group name; defaults to $Default

Custom endpoint address

Custom endpoint for Event Hubs (e.g., for private link); optional

Start from enqueue time

Optional enqueue time filter (ISO-8601); overrides starting position when set

Default50

Max batch size per partition

Maximum events pulled per partition read; defaults to 50

DefaultPT10S

Overall max duration

Stop consuming after this duration; defaults to PT10S

DefaultPT5S

Max wait per partition

Maximum wait for a partition batch before returning; defaults to PT5S

DefaultEARLIEST
Possible Values
EARLIESTLATESTINSTANT

Starting position

Initial position strategy per partition; defaults to EARLIEST

Reference (ref) of the pluginDefaults to apply to this task.

SAS token

Precomputed SAS token for Event Hubs; optional alternative to shared key/connection string

Shared key

Access key paired with sharedKeyAccountName; ignored if connectionString is provided

Shared key account name

Event Hubs namespace name used with sharedKeyAccountAccessKey when no connection string is provided

Events consumed

Formaturi

Consumed events URI

kestra:// URI for Ion file containing consumed events

The total number of events consumed.

More Plugins in this Category

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks

PostgreSQL
PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Data

6 tasks

25 blueprints