Ask a local knowledge base over HTTP, with retrieved sources

A small, private "ask our docs" API. Send a question to a webhook and get back an answer written by a local LLM from your own documents, together with the files of the passages it was given. If no document is relevant, the question never reaches the LLM and the API says so, instead of letting a small model make something up.

It reads the Qdrant collection maintained by local-rag-knowledge-base-sync, which keeps the index in step with a MinIO bucket.

This blueprint was created by kruxarth.

How it works

1. ask_over_http (Webhook) starts the flow with a JSON body such as {"question": "..."}. With wait and returnOutputs, the HTTP call blocks until the answer is ready and returns the flow outputs. A manual run uses the question input instead.
2. retrieve_context (io.kestra.plugin.ai.rag.Search) embeds the question with nomic-embed-text and looks for chunks with a relevance of at least 0.75 in Qdrant.
3. answer_or_abstain (If):
   • At least one relevant chunk: answer (io.kestra.plugin.ai.rag.ChatCompletion) asks qwen2.5:3b to answer from the retrieved chunks only, and log_answer logs the answer with its sources.
   • No relevant chunk: log_abstain logs a warning and the LLM is not called.
4. The flow outputs answer and sources, the unique source keys of the chunks given to the model.

What you get

curl -X POST http://localhost:8080/api/v1/main/executions/webhook/company.ai/local-rag-knowledge-base-ask/<KB_WEBHOOK_KEY> \
  -H 'Content-Type: application/json' -d '{"question": "How many weeks of parental leave do primary caregivers get?"}'

{"answer": "20 weeks", "sources": ["hr/employee-handbook.pdf"]}

An off-topic question returns {"answer": "The knowledge base does not cover this question.", "sources": []} in under a second, without calling the LLM. On a laptop CPU, answers took 3 to 9 seconds in testing, plus about 20 seconds for the first question while Ollama loads the model.

Who it's for

• Teams that want an internal Q&A endpoint for Slack bots, portals or scripts without sending documents to a hosted model.
• AI practitioners who want to see each retrieval and answer as an execution they can inspect.

Why orchestrate this with Kestra

• Every question is an execution: the retrieved chunks, the prompt, the answer and the token usage are all kept, which makes bad answers easy to debug.
• The abstain gate is a plain If on the retrieval result, so it is visible and easy to tune.
• Swapping the model, the threshold or the vector store is a change to YAML, not to application code.

Prerequisites

• The collection filled by local-rag-knowledge-base-sync, with the same Qdrant and Ollama services.
• The chat model: docker exec ollama ollama pull qwen2.5:3b.
• If Kestra runs in Docker, replace localhost in the variables with host.docker.internal.

Secrets

• QDRANT_API_KEY: the Qdrant API key, the same one as for the sync flow.
• KB_WEBHOOK_KEY: the key in the webhook URL. Anyone who knows it can query your documents, so keep it secret.

Quick start

1. Index a few documents with local-rag-knowledge-base-sync.
2. Execute this flow from the UI with a question your documents answer, then with one they do not, and compare the two executions.
3. Call the webhook with curl as shown above.

Things to know

• The 0.75 threshold is a LangChain4j relevance score, (cosine + 1) / 2. With nomic-embed-text on the test documents, the passage holding the answer scored between 0.72 and 0.92, off-topic questions at most 0.72, and passages from other documents on similar topics up to 0.82. So 0.75 refuses off-topic questions but also some very short ones (Do I need MFA? scored 0.72), and sources can list a related file that did not contribute to the answer. Measure it on your own documents, and again if you change the embedding model.
• qwen2.5:1.5b is faster but was easily distracted by unrelated chunks. qwen2.5:3b is the smallest model that answered our test questions reliably.

How to extend

• Call the webhook from a Slack slash command or a chat UI.
• Add memory to answer for follow-up questions.
• Log unanswered questions to a KV entry or a table to find gaps in the documentation.

Links

• AI plugin: RAG ChatCompletion
• AI plugin: Search
• Webhook trigger
• Ollama models