Source
yaml
id: redis-list-realtime-trigger
namespace: company.team
tasks:
- id: insert_into_cassandra
type: io.kestra.plugin.cassandra.Query
session:
endpoints:
- hostname: localhost
port: 9042
localDatacenter: datacenter1
cql: |
INSERT INTO kestra.products (product_id, product_name, product_category, brand)
VALUES ({{ trigger.value | jq(".product_id") | first }}, '{{ trigger.value | jq(".product_name") | first }}',
'{{ trigger.value | jq(".product_category") | first }}', '{{ trigger.value | jq(".brand") | first }}')
triggers:
- id: realtime_trigger
type: io.kestra.plugin.redis.list.RealtimeTrigger
url: redis://localhost:6379/0
key: products
About this blueprint
Trigger Realtime Trigger
This flow will:
- Get triggered every time you push data onto Redis List
- The flow will push the data into a table in Cassandra
To setup Cassandra server locally, use the following docker command:
text
docker run --name my-cassandra -p 9042:9042 -d cassandra
You can use the cqlsh in the Cassandra docker container, and run the following commands:
text
# Create the keyspace
> create keyspace if not exists kestra with replication = {'class' :
'SimpleStrategy', 'replication_factor' : 1};
# Use the keyspace
> use kestra;
# Create the table
> CREATE TABLE kestra.products (
product_id int,
product_name text,
product_category text,
brand text,
PRIMARY KEY (product_id));
To setup Redis locally, use the following Docker command:
text
docker run --name my-redis -p 6379:6379 -d redis
You can use the redis-cli in the Redis docker container, and push data onto Redis using:
text
> LPUSH products '{"product_id": 1, "product_name": "streamline turn-key
systems", "product_category": "Electronics", "brand": "gomez"}'
We will be using the JSON records generated from the data in products.csv.
Whenever you push the data onto Redis List, the flow will be triggered immediately, and will insert the data from the trigger into Cassandra table.
More Related Blueprints