RealtimeTrigger
Consume a message in real-time from Pulsar topics and create one execution per message.
If you would like to consume multiple messages processed within a given time frame and process them in batch, you can use the io.kestra.plugin.pulsar.Trigger instead.
type: "io.kestra.plugin.pulsar.RealtimeTrigger"Consume a message from a Pulsar topic in real-time.
id: pulsar
namespace: company.team
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "{{ trigger.value }}"
triggers:
- id: realtime_trigger
type: io.kestra.plugin.pulsar.RealtimeTrigger
topic: kestra_trigger
uri: pulsar://localhost:26650
deserializer: JSON
subscriptionName: kestra_trigger_sub
Deserializer used for the value.
The position of a subscription to the topic.
The subscription name.
Using subscription name, we will fetch only records that haven't been consumed yet.
The subscription type.
Pulsar topic(s) where to consume messages from.
Can be a string or a list of strings to consume from multiple topics.
Connection URLs.
You need to specify a Pulsar protocol URL.
- Example of localhost:
pulsar://localhost: 6650 - If you have multiple brokers:
pulsar://localhost: 6650,localhost: 6651,localhost: 6652 - If you use TLS authentication:
pulsar+ssl://pulsar.us-west.example.com: 6651
Authentication token.
Authentication token that can be required by some providers such as Clever Cloud.
The consumer name.
Add all the properties in the provided map to the consumer.
Add a public encryption key to the producer/consumer.
JSON string of the topic's schema
Required for connecting with topics with a defined schema and strict schema checking
The schema type of the topic
Can be one of NONE, AVRO or JSON. None means there will be no schema enforced.
List of execution states after which a trigger should be stopped (a.k.a. disabled).
TLS authentication options.
You need to use "pulsar+ssl://" in serviceUrl to enable TLS support.
The ca certificate.
Must be a base64-encoded pem file.
The client certificate.
Must be a base64-encoded pem file.
The key certificate.
Must be a base64-encoded pem file.