About this blueprint
Trigger Realtime Trigger SQL
This flow will:
- Get triggered every time the event lands in Apache Pulsar topic
- The flow will push the data into a table in MySQL database
To setup Apache Pulsar locally, you can install the standalone cluster or docker cluster for Apache Pulsar. You can run the following commands to create the topic, and produce data to the topic:
- Setup a tenant
bin/pulsar-admin tenants create apache - Create a namespace
bin/pulsar-admin namespaces create apache/pulsar - Create a topic
bin/pulsar-admin topics create-partitioned-topic apache/pulsar/logs -p 4 - Produce data to topic
bin/pulsar-client produce apache/pulsar/logs -m '--Hello World--' -n 1
To setup MySQL server locally, follow the official installation steps using docker.
yaml
id: pulsar-realtime-trigger
namespace: company.team
tasks:
- id: create_mysql_table
type: io.kestra.plugin.jdbc.mysql.Query
sql: |
create table if not exists logs (
log_id INT AUTO_INCREMENT,
message VARCHAR(500),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY(log_id)
)
- id: insert_into_logs_table
type: io.kestra.plugin.jdbc.mysql.Query
sql: insert into logs(message) values("{{ trigger.value }}")
triggers:
- id: realtime_trigger
type: io.kestra.plugin.pulsar.RealtimeTrigger
topic: apache/pulsar/logs
uri: pulsar://localhost:26650
subscriptionName: kestra_trigger_sub
pluginDefaults:
- type: io.kestra.plugin.jdbc.mysql.Query
values:
url: jdbc:mysql://localhost:3306/kestra
username: mysql_user
password: mysql_passwd