Build a Chat-With-Your-Data System Using Elasticsearch and OpenAI (RAG)

Build a chat-with-your-data system that answers natural-language questions over your own documents using the Retrieval-Augmented Generation (RAG) pattern. This Kestra blueprint wires Elasticsearch full-text retrieval to an OpenAI large language model so users get factual, grounded answers instead of hallucinations. It solves the core RAG problem: an LLM alone does not know your private knowledge base, and a search index alone returns raw documents rather than a clean answer. Together, keyword retrieval plus generation turns an FAQ index into a conversational assistant for support, documentation, and internal knowledge.

How it works

1. The search task (io.kestra.plugin.elasticsearch.Search) runs a bool query against the course_questions index, using a multi_match over the question, text, and section fields and a term filter on course. It returns the top five matching documents for the user's question.
2. The context_template task (io.kestra.plugin.core.debug.Return) loops over outputs.search.rows with a Pebble for block to assemble the retrieved sections, questions, and text into a single context string.
3. The generate_response task (io.kestra.plugin.openai.ChatCompletion) sends a teaching-assistant prompt with the user question and the built context to the gpt-4o model, capped at maxTokens: 500, instructed to answer only from context and return NONE when the answer is absent.
4. The log_output task (io.kestra.plugin.core.log.Log) extracts the generated answer from choices with a jq expression and logs it.

Two inputs drive each run: a free-text question and a select_a_zoomcamp SELECT that scopes retrieval to one course.

What you get

• Grounded answers sourced strictly from your indexed documents
• A reusable retrieval-plus-generation pattern you can point at any index
• Scoped search via a filter input, so one flow serves many knowledge domains
• Clear, inspectable outputs at every step for debugging and tuning

Who it's for

• Data and ML engineers prototyping RAG over existing search infrastructure
• Platform teams building FAQ, documentation, or support assistants
• Developers who want an LLM grounded in private content without a vector database

Why orchestrate this with Kestra

Elasticsearch and OpenAI have no shared scheduler that can chain retrieval, prompt assembly, and generation into one governed pipeline. Kestra declares the whole flow in YAML, passes outputs between tasks with templated expressions, and gives you event triggers, retries on transient API or cluster errors, and full execution lineage with per-task logs and outputs. That is the gap the underlying tools cannot fill on their own: a single, observable, replayable definition of the entire RAG chain.

Prerequisites

• A reachable Elasticsearch cluster (defaults to http://localhost:9200/) with a course_questions index
• An OpenAI account and API key for the ChatCompletion task

Secrets

This example uses inline configuration rather than Kestra secrets: the Elasticsearch hosts value and the OpenAI apiKey are set directly on the tasks. For any real deployment, replace the placeholder apiKey with {{ secret('OPENAI_API_KEY') }} and store the value in your secret backend.

Quick start

1. Start Elasticsearch locally:

   docker run -it --rm --name elasticsearch -m 2G -p 9200:9200 -p 9300:9300 \
     -e "discovery.type=single-node" -e "xpack.security.enabled=false" \
     docker.elastic.co/elasticsearch/elasticsearch:8.14.3
   

2. Create the course_questions index:

   curl -X PUT "http://localhost:9200/course_questions" \
   -H "Content-Type: application/json" -d'
   { "mappings": { "properties": {
     "text": { "type": "text" }, "section": { "type": "text" },
     "question": { "type": "text" }, "course": { "type": "keyword" } } } }'
   

3. Load the sample FAQ data:

   curl -X POST "http://localhost:9200/course_questions/_bulk" \
   -H "Content-Type: application/json" \
   --data-binary @<(curl -s \
   https://huggingface.co/datasets/kestra/datasets/raw/main/json/zoomcamp_faq.json \
   | jq -c '.[] | {"index":{}}, .')
   

4. Set your OpenAI API key on the generate_response task, then execute the flow with a question.

How to extend

• Point indexes and the query fields at your own Elasticsearch index to chat with internal docs, tickets, or product content.
• Swap multi_match for a vector or hybrid query to add semantic retrieval.
• Add a webhook or schedule trigger to expose the flow as a question-answering endpoint.
• Capture the answer downstream: post to Slack, store in a database, or open a follow-up task when the model returns NONE.

Links

• Elasticsearch plugin docs: https://kestra.io/plugins/plugin-elasticsearch
• OpenAI plugin docs: https://kestra.io/plugins/plugin-openai

Thanks to Faithful Adeda for contributing this example.