Blueprints

Use Pulsar Realtime Trigger to push events into MySQL

Source

yaml
id: pulsar-realtime-trigger
namespace: company.team

tasks:
  - id: create_mysql_table
    type: io.kestra.plugin.jdbc.mysql.Query
    sql: |
      create table if not exists logs (
        log_id INT AUTO_INCREMENT,
        message VARCHAR(500),
        created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
        PRIMARY KEY(log_id)
      )

  - id: insert_into_logs_table
    type: io.kestra.plugin.jdbc.mysql.Query
    sql: insert into logs(message) values("{{ trigger.value }}")

triggers:
  - id: realtime_trigger
    type: io.kestra.plugin.pulsar.RealtimeTrigger
    topic: apache/pulsar/logs
    uri: pulsar://localhost:26650
    subscriptionName: kestra_trigger_sub

pluginDefaults:
  - type: io.kestra.plugin.jdbc.mysql.Query
    values:
      url: jdbc:mysql://localhost:3306/kestra
      username: mysql_user
      password: mysql_passwd
See more

About this blueprint

Trigger Realtime Trigger SQL

This flow will:

  1. Get triggered every time the event lands in Apache Pulsar topic
  2. The flow will push the data into a table in MySQL database To setup Apache Pulsar locally, you can install the standalone cluster or docker cluster for Apache Pulsar. You can run the following commands to create the topic, and produce data to the topic:
  1. Setup a tenant bin/pulsar-admin tenants create apache
  2. Create a namespace bin/pulsar-admin namespaces create apache/pulsar
  3. Create a topic bin/pulsar-admin topics create-partitioned-topic apache/pulsar/logs -p 4
  4. Produce data to topic bin/pulsar-client produce apache/pulsar/logs -m '--Hello World--' -n 1 To setup MySQL server locally, follow the official installation steps using docker.

Query

Realtime Trigger

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra