Blueprints

Use Redis List Realtime Trigger to push events into Cassandra

About this blueprint

Realtime

This flow will:

  1. Get triggered every time you push data onto Redis List
  2. The flow will push the data into a table in Cassandra

To setup Cassandra server locally, use the following docker command:

docker run --name my-cassandra -p 9042:9042 -d cassandra

You can use the cqlsh in the Cassandra docker container, and run the following commands:

# Create the keyspace
> create keyspace if not exists kestra with replication = {'class' : 'SimpleStrategy', 'replication_factor' : 1};

# Use the keyspace
> use kestra;

# Create the table
>CREATE TABLE kestra.products (
                 product_id int, 
                 product_name text, 
                 product_category text, 
                 brand text, 
                 PRIMARY KEY (product_id));

To setup Redis locally, use the following Docker command:

docker run --name my-redis -p 6379:6379 -d redis

You can use the redis-cli in the Redis docker container, and push data onto Redis using:

> LPUSH products '{"product_id": 1, "product_name": "streamline turn-key systems", "product_category": "Electronics", "brand": "gomez"}'

We will be using the JSON records generated from the data in products.csv.

Whenever you push the data onto Redis List, the flow will be triggered immediately, and will insert the data from the trigger into Cassandra table. Whenever

yaml
id: redis_list_realtime_trigger
namespace: company.team

tasks:
  - id: insert_into_cassandra
    type: io.kestra.plugin.cassandra.Query
    session:
      endpoints:
        - hostname: localhost
          port: 9042
      localDatacenter: datacenter1
    cql: |
      INSERT INTO kestra.products (product_id, product_name, product_category, brand)
      VALUES ({{ trigger.value | jq(".product_id") | first }}, '{{ trigger.value | jq(".product_name") | first }}',
      '{{ trigger.value | jq(".product_category") | first }}', '{{ trigger.value | jq(".brand") | first }}')

triggers:
  - id: realtime_trigger
    type: io.kestra.plugin.redis.list.RealtimeTrigger
    url: redis://localhost:6379/0
    key: products

Query

Realtime Trigger

More Related Blueprints

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra