Reduce LLM API Costs with a Redis Semantic Cache for Repeated Questions

Cut OpenAI API spend and response latency by putting a Redis caching layer in front of your LLM. This blueprint normalizes each incoming question into a deterministic cache key, looks it up in Redis with io.kestra.plugin.redis.string.Get, and serves the stored answer instantly when it exists. Only on a true cache miss does it call the model, then writes the fresh answer back to Redis for next time. It solves the classic FAQ problem where users ask the same thing in slightly different wording and you pay for the same completion again and again.

How it works

1. The generate_cache_key task (io.kestra.plugin.scripts.python.Script) normalizes the question input (lowercase, strip punctuation, collapse whitespace) and MD5-hashes it with a cache_key_prefix, so semantically equivalent phrasings map to one key. It returns cache_key and normalized_question via Kestra.outputs.
2. check_cache (io.kestra.plugin.redis.string.Get) reads that key from Redis with failedOnMissing: false, so a miss returns null instead of erroring.
3. route_request (io.kestra.plugin.core.flow.If) branches on whether the cached value is present and non-empty.
4. On a hit, serve_from_cache logs the stored answer with no model call. On a miss, call_llm (io.kestra.plugin.openai.ChatCompletion, gpt-4o) generates an answer, store_in_cache (io.kestra.plugin.redis.string.Set) persists it, and log_llm_response logs the result.

What you get

• Instant responses for repeated questions, served straight from Redis
• Lower OpenAI bills: the model runs only on genuine cache misses
• A reusable, tenant-aware key scheme via cache_key_prefix
• Full execution lineage showing every hit and miss

Who it's for

• Teams running customer support or FAQ chatbots on top of an LLM
• Platform and AI engineers optimizing token cost and latency
• Anyone serving documentation or internal Q&A assistants over a bounded domain

Why orchestrate this with Kestra

Redis is a cache, not a scheduler: it cannot trigger on events, retry a failed model call, or give you visibility into why a question hit or missed. Kestra wraps the whole pattern in declarative YAML, adds event triggers and retries, branches with io.kestra.plugin.core.flow.If, and records execution lineage across the Python, Redis, and OpenAI tasks so the cache layer is observable and reproducible.

Prerequisites

• A reachable Redis instance
• An OpenAI API key with access to gpt-4o

Secrets

• REDIS_URL: connection URL for your Redis instance
• OPENAI_API_KEY: OpenAI API key for the chat completion call

Quick start

1. Add the REDIS_URL and OPENAI_API_KEY secrets to your namespace.
2. Add this flow to a Kestra namespace.
3. Execute it with the default question, or supply your own and a cache_key_prefix.
4. Run it twice with the same question and confirm the second execution is a cache hit.

How to extend

• Set a TTL on store_in_cache so cached answers expire and refresh.
• Add a vector similarity step so near-duplicate questions reuse one answer.
• Track hit and miss rates and push them to a monitoring dashboard.
• Swap the model or add a per-tenant cache_key_prefix for multi-tenant isolation.

Links

• Redis plugin documentation
• Redis orchestration with Kestra