Redis is a fast data store and event bus, not a workflow engine. Kestra adds the layer it lacks: schedule cache warming and invalidation, consume a Redis list as a real-time trigger, and coordinate Redis with your databases, APIs, and LLMs, all with retries and one execution history.
Redis stores, queues, and publishes data at millisecond latency, but it has no scheduler, no DAG, and no retry or replay layer. Kestra supplies all of it: it warms and invalidates caches on a Schedule, turns each item on a Redis list into a governed execution with RealtimeTrigger, and coordinates Redis with Postgres, APIs, and LLMs in one flow. Redis stays the fast layer; Kestra makes it part of a pipeline.
Redis runs the cache, the lists, and the pub/sub. Kestra runs the steps around the key: what triggers the flow, what feeds it, what runs next, and how Redis stays in sync with everything else.
Redis pub/sub and lists move messages fast, but cannot run a multi-step, multi-system pipeline per message. io.kestra.plugin.redis.list.RealtimeTrigger pops each item off a list and starts one execution per event, so a message becomes a governed run that fans out to Postgres, an API, or a notification, with retries and history.
Redis has TTL, but no cron, no dependency ordering, and no alert on failure. Kestra runs io.kestra.plugin.redis.string.Set and json.Set on a Schedule to warm a cache from the source of truth, and Delete or Ttl to invalidate it, with retries and a logged run the moment a warm fails.
Calling an LLM for a question you already answered is wasted spend. Kestra hashes the prompt, checks Redis with string.Get, and only calls the model on a true miss behind an If branch, writing the answer back with string.Set. Repeated questions return instantly from Redis instead of hitting the API again.
A cache is only useful when it matches the system of record. Kestra reads fresh rows from Postgres or another database, writes them to Redis as keys, and shares key-value state across flows, so the cache and the source stay consistent without hand-written sync scripts.
Loading thousands of keys one request at a time is slow. Kestra fans a JSON payload out with ForEach and writes each entry to Redis in parallel with string.Set, so a bulk seed or refresh finishes in one short, observable run instead of a serial loop you have to babysit.
Redis rarely works alone; it sits beside Kafka, AMQP, and model APIs. Kestra wires those together in one flow, consuming a Redis list and producing to a message broker, or caching embeddings between an LLM and a vector store, so Redis becomes one node in a real-time pipeline, not an island.
Patterns platform and data engineering teams run in production today. Each one shows the flow end to end, with the real plugin classes in play.
An upstream service pushes events onto a Redis list. The RealtimeTrigger pops each item the instant it lands and starts one execution that writes the record into a database, then confirms in Slack. Streaming ingestion with no polling and one run per event.
Each item popped off the list starts its own governed run with the payload.
RealtimeTrigger consumes the list continuously, not on a cron tick.
Swap the Cassandra task for Postgres or another store.
FAQ-style questions hit the same answers repeatedly. Kestra normalizes and hashes the prompt, checks Redis with string.Get, and only calls the model on a true miss behind an If, writing the result back with string.Set. Repeated questions return instantly instead of re-billing the API.
The model is called only on a genuine cache miss, not on every repeat.
Normalizing and hashing the prompt maps equivalent questions to one key.
A cached answer returns straight from Redis with no API round trip.
Seeding or refreshing a cache one key at a time is slow. Kestra takes a JSON payload, fans it out with ForEach, and writes each entry to Redis in parallel with string.Set, finishing a bulk load in one short run and pinging Discord when it completes.
ForEach writes keys concurrently instead of looping one request at a time.
Load a full key set from a single JSON input in one observable run.
Set options like mustNotExist and TTL control overwrite and expiry.
A cache drifts from the database unless something refreshes it. On a Schedule, Kestra reads the latest rows from Postgres, writes them to Redis with json.Set, and clears stale entries with Delete, so reads stay fast and current without a hand-written cron script.
Each run refreshes Redis from the system of record so reads stay current.
Delete clears stale keys as part of the same warming run.
A failed refresh logs and notifies, instead of silently serving stale data.
Flows often need a small piece of shared state: a cursor, a feature flag, a last-run marker. Kestra writes a JSON value to Redis with string.Set and reads it back later with string.Get, giving separate flows a fast, central place to coordinate without a database.
serdeType: JSON stores and retrieves structured objects, not just strings.
One flow writes a cursor or flag; another reads it on its next run.
Shared state lives in memory, so reads and writes return in milliseconds.
| Capability | ||||
|---|---|---|---|---|
| Trigger a pipeline from a Redis list or stream | Native RealtimeTrigger | Custom worker code | Custom sensor | Redis trigger node |
| Schedule cache warming and invalidation | Schedule + Set/Delete tasks | Code your own | Cron + Python operators | Cron + nodes |
| Semantic LLM cache with Redis | Get, If, and model tasks | Hand-coded activities | Python operators | Function nodes |
| Coordinate Redis with databases and APIs | Native plugins | Activity code per service | Operators, Python required | One node per service |
| Parallel bulk key writes | ForEach + Set | Code your own | Dynamic tasks, Python | Split and loop nodes |
| Declarative YAML, no code required | Pure YAML | Go, Java, or TS code | Python DAGs | Visual + JS functions |
| Retries, backfills, replay from failure | Per-task, built in | Yes, in code | Yes, Python config | Limited |
Find answers to your questions right here, and don't hesitate to Contact Us if you couldn't find what you're looking for.
Consume lists as real-time triggers, warm and invalidate caches on a schedule, build a semantic LLM cache, and coordinate Redis with your databases and APIs in one flow. Open source, self-hosted, event-driven.
