List icon
Get icon
Request icon
Search icon
OutputValues icon
If icon
SlackIncomingWebhook icon
Log icon
Pause icon
Fail icon
Toggle icon
ResetCheckpoint icon
Sleep icon
Schedule icon

Quickwit Source Checkpoint Reset Recovery Guard

Detect stale Quickwit Kafka indexing and verify recovery after an approved checkpoint reset.

Categories
Data

Safety

Resetting a source checkpoint can replay a Kafka topic into an index. This flow names one index and one source, reads the checkpoint again after approval, disables the source before resetting it and re-reads the source before every write. It reports recovery only when the checkpoint changed and Quickwit search returns a document newer than the original freshness budget.

Guard What it stops
auto_remediate plus dry_run Any source write in the default schedule mode
Pause with approved ?? false An unapproved Toggle or ResetCheckpoint
Checkpoint and source re-read before each write Acting on a source that changed during approval or an earlier write
Named index_id and source_id A bulk reset across unrelated indexes or sources
Post-write checkpoint and search checks Reporting success when reset or indexing did not recover

How it works

  1. list_indexes (io.kestra.plugin.quickwit.index.List) confirms the target index exists, while read_index (io.kestra.plugin.quickwit.index.Get) reads its attached sources and checkpoint.
  2. read_latest and read_fresh (io.kestra.plugin.quickwit.search.Search) read the newest document and count documents at the freshness watermark. An enabled named source with zero fresh hits and a checkpoint is classified as degraded. Healthy, missing-source, disabled, and missing-index cases are spared.
  3. approval_gate (io.kestra.plugin.core.flow.Pause) runs only when auto_remediate is true and dry_run is false. A declined or unanswered pause logs and performs no write.
  4. After approval, read_before_disable checks the checkpoint and enabled source state. The flow calls source.Toggle with enable: false, reads again, calls source.ResetCheckpoint, reads again, then calls source.Toggle with enable: true.
  5. wait_for_recovery (io.kestra.plugin.core.flow.Sleep) bounds the recovery window. verify_index (index.Get) and verify_freshness (search.Search) require a changed checkpoint and at least one fresh searchable document or fail the execution.
  6. The errors block sends the execution id to Slack on an unhandled failure.

What you get

  • A detector for the silent failure mode where old Quickwit results still look healthy.
  • A scoped, approval-gated reset that handles a recreated or trimmed Kafka topic.
  • A post-change proof based on Quickwit checkpoint state and searchable document freshness.
  • Healthy, disabled-source, missing-source and stale-plan cases recorded in Kestra history.

Who it's for

Search and data platform teams running Quickwit Kafka sources that must keep dashboards and operational searches within a freshness budget after topic maintenance.

Why orchestrate this with Kestra

Quickwit's REST API can toggle and reset a source, but it does not provide an approval record, stale-plan protection, bounded recovery wait, Slack failure path or an auditable before and after execution. Kestra supplies those controls while every read and write still uses the Quickwit plugin.

Tested end to end

The test fixture uses Quickwit 0.8.2, Kafka 7.6.1, Kestra 2.0.5 and Quickwit plugin 1.0.0. See the evidence packet for execution ids, independent REST assertions and the second-fixture audit.

Case Execution Result
Healthy control UYqh96SbuNjBgcvQhQ3Ms SUCCESS, one fresh hit, no writes
Degraded dry run 3Cf1iBe6f5jK6OnO7ftNRY SUCCESS, zero fresh hits, no writes
Declined approval 2vdohVf4xD7kHs9T9YmvKF SUCCESS, no writes
Approved recovery 7K2HDM4IRrKnxkgGbKqWdD SUCCESS, checkpoint 18 to 19, one fresh hit
Repeat 7ikjFo1RIIsVfNG21dwvXm SUCCESS, healthy and no writes
Disabled source 3xFk6R7vDKeUrFoyqvToEy SUCCESS, source spared
Approval race 5KnQlBHsTL62FICwNxAf3s FAILED before Toggle
Partial interruption 5004v6AVbCwJrMYBHqF03m FAILED after Toggle and before reset

Prerequisites

  • Kestra 2.0.5 with io.kestra.plugin:plugin-quickwit:1.0.0 loaded.
  • Quickwit 0.8.x with a Kafka source and an index timestamp field named ts.
  • Kafka permissions to read the named topic and Quickwit permissions to read index metadata, search, toggle the source and reset its checkpoint.

Secrets

  • SLACK_WEBHOOK_URL: Slack Incoming Webhook used by errors and the stale detector alert.

Inputs

Input Default Purpose
quickwit_url http://localhost:7280 Quickwit REST base URL
index_id logs One target index
source_id source One named Kafka source
freshness_minutes 5 Freshness watermark window
wait_seconds 20 Bounded post-reset wait
auto_remediate false Enables the approval branch
dry_run true Keeps writes disabled

Quick start

  1. Load the Quickwit plugin into the Kestra 2.0.5 runner and set SLACK_WEBHOOK_URL.
  2. Set quickwit_url, index_id and source_id to the real target.
  3. Run with auto_remediate: false and dry_run: true to inspect the detector.
  4. If the named source is degraded, rerun with auto_remediate: true, dry_run: false, approve the Pause and wait for the recovery proof.

How to extend

Add a per-index freshness budget only when the index timestamp semantics are equivalent to ts and keep each source reset behind its own approval. The flow does not prove Kafka broker health or guarantee that every historical document is replayed before the wait expires.

Limits: a source whose checkpoint is already advancing but whose documents are malformed can still fail the freshness check; the flow reports that failure and leaves the source enabled.

Links

This blueprint was created by zkasuran.

See How

New to Kestra?

Use blueprints to kickstart your first workflows.