Meilisearch Synonym Relevance Gate

id: meilisearch-synonym-dictionary-relevance-gate
namespace: company.team

description: |
  Validate and promote search synonym dictionaries in Meilisearch without risking relevance regressions.
  Stages proposed synonym rules against an isolated canary index, runs automated recall and
  category pollution queries, and promotes to the production index only when quality gates pass.

inputs:
  - id: meilisearch_url
    type: STRING
    displayName: Meilisearch URL
    description: Base URL of the Meilisearch server or cluster endpoint.
    defaults: "http://localhost:7700"

  - id: production_index
    type: STRING
    displayName: Production Index UID
    description: Identifier of the live production index where verified synonyms are
      promoted.
    defaults: "products"

  - id: canary_synonym_query
    type: STRING
    displayName: Canary Synonym Query
    description: Search term present in the staged synonym dictionary to verify
      recall expansion.
    defaults: "trainers"

  - id: canary_min_hits
    type: INT
    displayName: Canary Minimum Hits
    description: Minimum number of document matches expected from the synonym query.
    defaults: 1

  - id: canary_control_query
    type: STRING
    displayName: Canary Control Query
    description: Unrelated keyword query to verify that synonym expansion causes no
      cross-category pollution.
    defaults: "mechanical keyboard"

  - id: canary_facet_name
    type: STRING
    displayName: Canary Facet Name
    description: Facet attribute used to verify distribution integrity after synonym
      expansion.
    defaults: "category"

  - id: auto_promote
    type: BOOL
    displayName: Auto Promote Synonyms
    description: If true, automatically updates the production index settings upon
      passing canary tests.
    defaults: true

triggers:
  - id: synonym_update_webhook
    type: io.kestra.plugin.core.trigger.Webhook
    description: Webhook endpoint for merchandising tools or GitOps pipelines to
      trigger synonym testing.
    key: "{{ secret('MEILISEARCH_SYNONYM_WEBHOOK_KEY') }}"

  - id: weekly_dictionary_audit
    type: io.kestra.plugin.core.trigger.Schedule
    description: Scheduled off-peak maintenance audit of synonym dictionaries.
      Disabled by default.
    cron: "0 4 * * 1"
    disabled: true

tasks:
  - id: stage_canary_index
    type: io.kestra.plugin.core.http.Request
    description: Initialize an isolated canary index with an explicit primary key
      for synonym evaluation.
    uri: "{{ inputs.meilisearch_url }}/indexes"
    method: POST
    contentType: application/json
    headers:
      Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"
    body: |
      {
        "uid": "{{ inputs.production_index }}_synonym_canary",
        "primaryKey": "id"
      }
    options:
      allowFailed: true
    allowFailure: true

  - id: configure_canary_attributes
    type: io.kestra.plugin.core.http.Request
    description: Configure filterable attributes on the canary index to enable
      faceted relevance queries.
    uri: "{{ inputs.meilisearch_url }}/indexes/{{ inputs.production_index
      }}_synonym_canary/settings/filterable-attributes"
    method: PUT
    contentType: application/json
    headers:
      Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"
    body: |
      [
        "category",
        "brand",
        "in_stock"
      ]

  - id: prepare_sample_corpus
    type: io.kestra.plugin.scripts.python.Script
    description: Prepare a controlled product dataset covering target and control
      categories for relevance testing.
    outputFiles:
      - corpus.json
    script: |
      import json

      documents = [
          {
              "id": "item-101",
              "title": "Pro Carbon Running Shoes",
              "description": "High-performance marathon running shoes with responsive foam midsole.",
              "category": "Footwear",
              "brand": "SwiftTrack",
              "price": 180.00,
              "in_stock": True
          },
          {
              "id": "item-102",
              "title": "Trail Grip Athletic Sneakers",
              "description": "Durable all-terrain outdoor sneakers designed for trail runners and hikers.",
              "category": "Footwear",
              "brand": "SwiftTrack",
              "price": 145.00,
              "in_stock": True
          },
          {
              "id": "item-103",
              "title": "Ultra RGB Mechanical Keyboard",
              "description": "Custom mechanical keyboard with linear switches and sound-dampening foam.",
              "category": "Electronics",
              "brand": "KeyCraft",
              "price": 129.99,
              "in_stock": True
          },
          {
              "id": "item-104",
              "title": "Ergonomic Optical Gaming Mouse",
              "description": "Lightweight gaming mouse with optical switches and high-precision sensor.",
              "category": "Electronics",
              "brand": "KeyCraft",
              "price": 69.50,
              "in_stock": True
          },
          {
              "id": "item-105",
              "title": "Waterproof Sports Duffel Bag",
              "description": "Compact gym duffel bag with dedicated ventilated sneaker compartment.",
              "category": "Accessories",
              "brand": "SwiftTrack",
              "price": 55.00,
              "in_stock": False
          }
      ]

      with open("corpus.json", "w", encoding="utf-8") as f:
          json.dump(documents, f, indent=2)

      print(f"Prepared {len(documents)} validation documents in corpus.json")

  - id: index_canary_documents
    type: io.kestra.plugin.meilisearch.DocumentAdd
    description: Ingest validation documents into the canary index and wait for
      asynchronous task indexing to complete.
    url: "{{ inputs.meilisearch_url }}"
    key: "{{ secret('MEILISEARCH_API_KEY') }}"
    index: "{{ inputs.production_index }}_synonym_canary"
    from: "{{ outputs.prepare_sample_corpus.outputFiles['corpus.json'] }}"
    waitForIndexing: true

  - id: apply_staged_synonyms
    type: io.kestra.plugin.core.http.Request
    description: Apply the proposed synonym dictionary to the canary index for evaluation.
    uri: "{{ inputs.meilisearch_url }}/indexes/{{ inputs.production_index
      }}_synonym_canary/settings/synonyms"
    method: PUT
    contentType: application/json
    headers:
      Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"
    body: |
      {
        "trainers": ["running shoes", "sneakers"],
        "sneakers": ["running shoes", "trainers"],
        "display": ["monitor", "screen"]
      }

  - id: canary_synonym_recall_search
    type: io.kestra.plugin.meilisearch.Search
    description: Verify that searching for the proposed synonym term successfully
      recalls target documents.
    url: "{{ inputs.meilisearch_url }}"
    key: "{{ secret('MEILISEARCH_API_KEY') }}"
    index: "{{ inputs.production_index }}_synonym_canary"
    query: "{{ inputs.canary_synonym_query }}"

  - id: canary_control_bleed_search
    type: io.kestra.plugin.meilisearch.Search
    description: Verify that control search queries for unrelated products are
      unaffected by synonym rules.
    url: "{{ inputs.meilisearch_url }}"
    key: "{{ secret('MEILISEARCH_API_KEY') }}"
    index: "{{ inputs.production_index }}_synonym_canary"
    query: "{{ inputs.canary_control_query }}"

  - id: canary_facet_distribution_search
    type: io.kestra.plugin.meilisearch.FacetSearch
    description: Validate that facet distributions on the target category remain
      accurate under synonym expansion.
    url: "{{ inputs.meilisearch_url }}"
    key: "{{ secret('MEILISEARCH_API_KEY') }}"
    index: "{{ inputs.production_index }}_synonym_canary"
    facetName: "{{ inputs.canary_facet_name }}"
    facetQuery: "Footwear"

  - id: evaluate_relevance_gate
    type: io.kestra.plugin.core.flow.If
    description: Gate synonym promotion on satisfying recall hit thresholds and
      maintaining control query precision.
    condition: "{{ (outputs.canary_synonym_recall_search.totalHits >=
      inputs.canary_min_hits) and (outputs.canary_control_bleed_search.totalHits
      >= 1) and (outputs.canary_facet_distribution_search.totalHits >= 1) }}"
    then:
      - id: evaluate_promotion
        type: io.kestra.plugin.core.flow.If
        description: Check whether automated promotion to the live production index is
          enabled.
        condition: "{{ inputs.auto_promote }}"
        then:
          - id: promote_synonyms_to_production
            type: io.kestra.plugin.core.http.Request
            description: Promote verified synonym rules to the live production search index.
            uri: "{{ inputs.meilisearch_url }}/indexes/{{ inputs.production_index
              }}/settings/synonyms"
            method: PUT
            contentType: application/json
            headers:
              Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"
            body: |
              {
                "trainers": ["running shoes", "sneakers"],
                "sneakers": ["running shoes", "trainers"],
                "display": ["monitor", "screen"]
              }

          - id: log_promotion_success
            type: io.kestra.plugin.core.log.Log
            description: Record successful synonym promotion details in execution logs.
            message: >-
              PROMOTED: Synonyms successfully validated and promoted to index
              '{{ inputs.production_index }}' (Synonym hits: {{
              outputs.canary_synonym_recall_search.totalHits }}, Control hits:
              {{ outputs.canary_control_bleed_search.totalHits }}, Facet hits:
              {{ outputs.canary_facet_distribution_search.totalHits }}).

        else:
          - id: log_dry_run_verdict
            type: io.kestra.plugin.core.log.Log
            description: Log that the synonym rules passed all validation gates without
              modifying production.
            message: >-
              CANARY PASSED: Synonyms validated successfully
              (auto_promote=false). Production index '{{ inputs.production_index
              }}' was untouched.

      - id: cleanup_canary_index
        type: io.kestra.plugin.core.http.Request
        description: Delete the canary index to reclaim cluster storage capacity.
        uri: "{{ inputs.meilisearch_url }}/indexes/{{ inputs.production_index
          }}_synonym_canary"
        method: DELETE
        headers:
          Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"

    else:
      - id: abort_and_delete_canary_index
        type: io.kestra.plugin.core.http.Request
        description: Relevance validation failed. Purge the canary index and abort
          production deployment.
        uri: "{{ inputs.meilisearch_url }}/indexes/{{ inputs.production_index
          }}_synonym_canary"
        method: DELETE
        headers:
          Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"

      - id: alert_relevance_regression
        type: io.kestra.plugin.core.log.Log
        description: Log error audit detailing why the staged synonyms failed validation.
        level: ERROR
        message: >-
          REJECTED: Staged synonyms failed relevance validation (Synonym hits:
          {{ outputs.canary_synonym_recall_search.totalHits }}/{{
          inputs.canary_min_hits }}, Control hits: {{
          outputs.canary_control_bleed_search.totalHits }}, Facet hits: {{
          outputs.canary_facet_distribution_search.totalHits }}). Production
          index was protected.

errors:
  - id: emergency_cleanup_canary
    type: io.kestra.plugin.core.http.Request
    description: Safety task ensuring canary index deletion if an unhandled error
      terminates the flow.
    uri: "{{ inputs.meilisearch_url }}/indexes/{{ inputs.production_index
      }}_synonym_canary"
    method: DELETE
    options:
      allowFailed: true
    allowFailure: true
    headers:
      Authorization: "Bearer {{ secret('MEILISEARCH_API_KEY') }}"

  - id: log_pipeline_failure
    type: io.kestra.plugin.core.log.Log
    description: Record fatal pipeline failure in execution logs.
    level: ERROR
    message: >-
      FATAL: Synonym relevance gate failed during execution {{ execution.id }}.
      Production index settings were untouched.

Search synonym dictionaries are critical for e-commerce conversion, enabling shoppers to find products across regional dialects and colloquialisms (such as mapping trainers to sneakers or running shoes). However, introducing unverified or overly broad synonym mappings directly into production search settings can severely degrade relevance, cause category bleed, or dilute ranking for high-value keywords.

This blueprint implements an automated staging and relevance regression gate for Meilisearch. It spins up an isolated canary index ({{ inputs.production_index }}_synonym_canary), ingests representative multi-category catalog documents via io.kestra.plugin.meilisearch.DocumentAdd, and applies the proposed synonym dictionary via Meilisearch's settings API. It then executes automated validation queries using io.kestra.plugin.meilisearch.Search and io.kestra.plugin.meilisearch.FacetSearch to verify both positive recall expansion and negative category isolation. Only when all canary checks pass does Kestra promote the synonyms to the live production index. If regressions are detected, the canary index is purged and production remains completely protected.

How it works

stage_canary_index (io.kestra.plugin.core.http.Request) creates an isolated staging index with the primary key id.
configure_canary_attributes (io.kestra.plugin.core.http.Request) sets filterable attributes (category, brand, in_stock) required for faceted query verification.
prepare_sample_corpus (io.kestra.plugin.scripts.python.Script) writes a representative multi-category product dataset to corpus.json in Kestra internal storage.
index_canary_documents (io.kestra.plugin.meilisearch.DocumentAdd) bulk-indexes the documents into the canary index, setting waitForIndexing: true to ensure the asynchronous indexing task completes before search queries run.
apply_staged_synonyms (io.kestra.plugin.core.http.Request) submits the proposed synonym dictionary (trainers -> running shoes, sneakers) to the canary index.
canary_synonym_recall_search (io.kestra.plugin.meilisearch.Search) searches for the synonym query trainers, verifying that documents containing running shoes are successfully retrieved.
canary_control_bleed_search (io.kestra.plugin.meilisearch.Search) executes a control query for an unrelated keyword (mechanical keyboard) to verify that the synonym update caused no negative ranking bleed or cross-category contamination.
canary_facet_distribution_search (io.kestra.plugin.meilisearch.FacetSearch) checks that facet counts for Footwear accurately reflect the expanded result set.
evaluate_relevance_gate (io.kestra.plugin.core.flow.If) requires all three canary checks to pass: synonym recall hits, control query hits, and facet distribution hits.
- Then (Validation Passed): If auto_promote is true, promote_synonyms_to_production applies the verified dictionary to inputs.production_index, followed by cleanup_canary_index to reclaim cluster memory.
- Else (Validation Failed): Purges the canary index and emits a structured error log detailing the relevance breach, leaving the live index untouched.
The flow-level errors block guarantees automatic cleanup of the canary index if any task encounters an unhandled exception.

What you get

Automated regression testing for search synonym changes before customer-facing traffic is affected.
Dual validation covering both positive recall expansion and negative control isolation.
Facet distribution verification to ensure merchandising filters remain consistent.
Safe automated promotion to the live production index upon passing all canary gates.
Complete cleanup of temporary canary indices on both success and failure paths.

Who it's for

Search Platform Engineers managing Meilisearch or Algolia clusters.
E-commerce Merchandisers tuning search dictionaries, synonyms, and product discoverability.
Data Platform Teams building GitOps pipelines for search engine configuration management.

Why orchestrate this with Kestra

Ad-hoc curl scripts or manual updates to search engine settings offer no safety net: a typo in a synonym JSON payload can instantly degrade search relevance for thousands of active shoppers. Kestra provides declarative workflow orchestration that stages changes, runs automated canary suites, conditionally promotes approved settings, and rolls back cleanly on failure. Built-in secret masking protects your Meilisearch master API key, while webhook triggers allow instant integration with CMS or merchandising dashboards.

Prerequisites

A running Meilisearch instance (v1.6+) accessible over HTTP/HTTPS.
An API key with administrative permissions to modify index settings and documents.

Secrets

MEILISEARCH_API_KEY: Administrative API key or master key for Meilisearch authentication.
MEILISEARCH_SYNONYM_WEBHOOK_KEY: Secret authentication key for the event-driven webhook trigger.

Quick start

Start a local Meilisearch container with Docker:

docker run -d -p 7700:7700 \
  -v /tmp/meili_data:/meili_data \
  getmeili/meilisearch:v1.12 \
  --master-key="testMasterKey12345"

Create the production index with initial schema:

curl -X POST "http://localhost:7700/indexes" \
  -H "Authorization: Bearer testMasterKey12345" \
  -H "Content-Type: application/json" \
  -d '{"uid": "products", "primaryKey": "id"}'

Configure the required secrets in your Kestra namespace:
- MEILISEARCH_API_KEY: testMasterKey12345
- MEILISEARCH_SYNONYM_WEBHOOK_KEY: secret-webhook-key-67890
Execute the flow in Kestra and inspect the logs to verify the canary tests and synonym promotion.

How to extend

Connect prepare_sample_corpus to an upstream database extract or S3 catalog dump to test synonyms against production product datasets.
Add Slack or Microsoft Teams notification tasks inside the then and else blocks to notify the merchandising team of promotion outcomes.
Extend the dictionary payload in apply_staged_synonyms to accept dynamic JSON inputs passed through the webhook body.

Meilisearch Synonym Dictionary and Search Relevance Regression Gate

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Links

Tasks

Share this Blueprint

Create a Blueprint

New to Kestra?

Meilisearch Synonym Dictionary and Search Relevance Regression Gate

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Links

Tasks

Share this Blueprint

Create a Blueprint

More Data Blueprints

Enforce Retention on a Pinecone Namespace with a Weekly Purge

Load Precomputed Embeddings into Pinecone with a Daily Upsert Pipeline

Apache Pinot Real-Time Ingestion Freshness Sentinel

Build a point-in-time correct training set and block feature leakage

Spot-Check Pinecone Retrieval Quality with a Query by Vector Id

Execute GDPR Right-to-Erasure on Pinecone Vectors with Proof of Deletion

New to Kestra?