Meilisearch Index Rebuild Pipeline with Dated Indexes and Manual Swap

Reindexing in place is how search outages happen: half-written indexes serve broken results while the rebuild runs, and a failed rebuild leaves nothing to roll back to. This blueprint uses the blue-green pattern instead. Every run builds a brand new index named with the execution date, docs_site_20250101 style, from a fresh corpus download and a Python reshape. io.kestra.plugin.meilisearch.DocumentAdd waits until the new index is fully searchable, then Slack tells the operator to verify it and swap it live with Meilisearch's atomic swap-indexes API. The serving index is never touched mid-build.

How it works

1. download_corpus (io.kestra.plugin.core.http.Download) fetches the complete content corpus so the rebuild starts from the full source of truth.
2. reshape_corpus (io.kestra.plugin.scripts.python.Script on the Process task runner) writes one search document per line with id, title, and body fields, and emits doc_count through Kestra's output protocol.
3. rebuild_index (io.kestra.plugin.meilisearch.DocumentAdd) targets a new index named docs_site_{{ execution.startDate | date('yyyyMMdd') }}. With waitForIndexing: true and a PT10M timeout, the task only succeeds once every batch is indexed; documentsAdded carries the final count.
4. notify_operator posts the new index name and document count to Slack with the exact cutover instruction: verify a few queries, then call POST /swap-indexes with docs_site and the dated index. Meilisearch swaps them atomically, queries never see a mix, and the old content remains under the dated name as an instant rollback.
5. The errors block posts a distinct alert on failure, noting explicitly that the live index is unaffected.
6. A disabled-by-default Schedule trigger rebuilds weekly on Sunday at 04:00.

What you get

• Blue-green reindexing: users search the old index until the new one is verifiably complete.
• Dated index names that double as build history and rollback targets.
• A rebuild that fails safely, since nothing serves traffic until the operator swaps.
• A clear human checkpoint in the middle of an otherwise automated pipeline.

Who it's for

• Platform teams operating Meilisearch behind production documentation or content sites.
• Search operators who need schema changes, analyzer updates, or corpus migrations without downtime.
• Teams adding governance to search operations, with an explicit approval moment before cutover.

Why orchestrate this with Kestra

A safe reindex is a sequence with state: fetch, transform, build, wait, verify, hand off. Kestra expresses the whole sequence declaratively, carries the corpus between steps in internal storage, derives the dated index name from execution metadata, and records every rebuild in the execution history, so each dated index maps to exactly one auditable run.

Prerequisites

• A running Meilisearch instance with capacity for two copies of the index during cutover.
• A JSON endpoint or export of your full content corpus; the demo uses a public placeholder API.
• A Slack incoming webhook for the operator handoff and failure alerts.

Secrets

• MEILISEARCH_URL: Meilisearch connection URL, e.g. https://meilisearch.internal:7700.
• MEILISEARCH_KEY: Meilisearch API key with document write and index creation permissions.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the three secrets to your Kestra namespace.
2. Execute the flow and confirm Slack names a dated index with the expected document count.
3. Verify a few queries against the dated index, then call the swap-indexes API to make it live.
4. Point download_corpus at your real corpus and set disabled: false on the weekly trigger.

How to extend

• Automate the cutover by adding an http.Request task that calls POST /swap-indexes after a Pause task for manual approval, keeping the human checkpoint inside the flow.
• Run the search canary blueprint against the dated index before announcing it, turning verification into a task instead of an instruction.
• Delete indexes older than N rebuilds with a cleanup task to reclaim disk.
• Copy index settings such as filterable attributes onto the new index with an http.Request before indexing, so facets work immediately after the swap.

Links

• Meilisearch plugin documentation
• Meilisearch documentation