Azure Produce
CertifiedPublish events to Azure Event Hubs
Azure Produce
Publish events to Azure Event Hubs
Reads records from a file or list, serializes bodies, batches by size/count, and sends to a single Event Hub. Defaults: bodySerializer=STRING, maxEventsPerBatch=1000. Optionally set partitionKey or maxBatchSizeInBytes.
type: io.kestra.plugin.azure.eventhubs.ProduceExamples
Publish a file as events into Azure EventHubs.
id: azure_eventhubs_send_events
namespace: company.team
inputs:
- id: file
type: FILE
description: a CSV file with columns id, username, tweet, and timestamp
tasks:
- id: read_csv_file
type: io.kestra.plugin.serdes.csv.CsvToIon
from: "{{ inputs.file }}"
- id: transform_row_to_json
type: io.kestra.plugin.graalvm.js.FileTransform
from: "{{ outputs.read_csv_file.uri }}"
script: |
var result = {
"body": {
"username": row.username,
"tweet": row.tweet
}
};
row = result
- id: send_to_eventhub
type: io.kestra.plugin.azure.eventhubs.Produce
from: "{{ outputs.transform_row_to_json.uri }}"
eventHubName: my_eventhub
namespace: my_event_hub_namespace
connectionString: "{{ secret('EVENTHUBS_CONNECTION') }}"
maxBatchSizeInBytes: 4096
maxEventsPerBatch: 100
bodySerializer: "JSON"
bodyContentType: application/json
eventProperties:
source: kestra
Properties
eventHubName *string
Event Hub name
Target Event Hub entity
from *stringarrayobject
Structured data items, either as a map, a list of map, a URI, or a JSON string.
Structured data items can be defined in the following ways:
- A single item as a map (a document).
- A list of items as a list of maps (a list of documents).
- A URI, supported schemes are
kestrafor internal storage files,filefor host local files, andnsfilefor namespace files. - A JSON String that will then be serialized either as a single item or a list of items.
namespace *string
Namespace
Event Hubs namespace when using AAD or shared key auth
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150bodyContentType string
Body content type
MIME type placed on each event for downstream consumers
bodySerializer string
STRINGSTRINGBINARYIONJSONBody serializer
Serde used to serialize event bodies; defaults to STRING
bodySerializerProperties object
{}Serializer properties
Key/value options passed to the serializer
clientMaxRetries integerstring
5Client max retries
Max retry attempts for Event Hubs client operations; default 5
clientRetryDelay integerstring
500Retry delay (ms)
Delay between client retries in milliseconds; default 500
connectionString string
Event Hubs connection string
Namespace or Event Hub–level connection string; overrides key/sas fields when set
customEndpointAddress string
Custom endpoint address
Custom endpoint for Event Hubs (e.g., for private link); optional
eventProperties object
{}Event properties
Metadata properties applied to each event body
maxBatchSizeInBytes integerstring
Max batch size (bytes)
Maximum batch payload size; optional
maxEventsPerBatch integerstring
1000Max events per batch
Cap on events per batch; defaults to 1000
partitionKey string
Partition key
Routes all events in this run to the same partition using hash
sasToken string
SAS token
Precomputed SAS token for Event Hubs; optional alternative to shared key/connection string
Outputs
eventsCount integer
Events sent
sendBatchesCount integer
Batches sent
Metrics
batches.sent.count counter
The total number of batches sent.
events.sent.count counter
The total number of events sent.
