Clear Pulsar backlogs held by subscriptions nobody consumes
Detect Apache Pulsar subscriptions with no consumers that still pin a large backlog, then clear or remove them behind a dry run, an approval pause and a live re-read that fails the run.
Detect stale Quickwit Kafka indexing and verify recovery after an approved checkpoint reset.
Resetting a source checkpoint can replay a Kafka topic into an index. This flow names one index and one source, reads the checkpoint again after approval, disables the source before resetting it and re-reads the source before every write. It reports recovery only when the checkpoint changed and Quickwit search returns a document newer than the original freshness budget.
| Guard | What it stops |
|---|---|
auto_remediate plus dry_run |
Any source write in the default schedule mode |
Pause with approved ?? false |
An unapproved Toggle or ResetCheckpoint |
| Checkpoint and source re-read before each write | Acting on a source that changed during approval or an earlier write |
Named index_id and source_id |
A bulk reset across unrelated indexes or sources |
| Post-write checkpoint and search checks | Reporting success when reset or indexing did not recover |
list_indexes (io.kestra.plugin.quickwit.index.List) confirms the target index exists, while
read_index (io.kestra.plugin.quickwit.index.Get) reads its attached sources and checkpoint.read_latest and read_fresh (io.kestra.plugin.quickwit.search.Search) read the newest
document and count documents at the freshness watermark. An enabled named source with zero
fresh hits and a checkpoint is classified as degraded. Healthy, missing-source, disabled,
and missing-index cases are spared.approval_gate (io.kestra.plugin.core.flow.Pause) runs only when auto_remediate is true
and dry_run is false. A declined or unanswered pause logs and performs no write.read_before_disable checks the checkpoint and enabled source state. The
flow calls source.Toggle with enable: false, reads again, calls source.ResetCheckpoint,
reads again, then calls source.Toggle with enable: true.wait_for_recovery (io.kestra.plugin.core.flow.Sleep) bounds the recovery window.
verify_index (index.Get) and verify_freshness (search.Search) require a changed
checkpoint and at least one fresh searchable document or fail the execution.errors block sends the execution id to Slack on an unhandled failure.Search and data platform teams running Quickwit Kafka sources that must keep dashboards and operational searches within a freshness budget after topic maintenance.
Quickwit's REST API can toggle and reset a source, but it does not provide an approval record, stale-plan protection, bounded recovery wait, Slack failure path or an auditable before and after execution. Kestra supplies those controls while every read and write still uses the Quickwit plugin.
The test fixture uses Quickwit 0.8.2, Kafka 7.6.1, Kestra 2.0.5 and Quickwit plugin 1.0.0. See the evidence packet for execution ids, independent REST assertions and the second-fixture audit.
| Case | Execution | Result |
|---|---|---|
| Healthy control | UYqh96SbuNjBgcvQhQ3Ms |
SUCCESS, one fresh hit, no writes |
| Degraded dry run | 3Cf1iBe6f5jK6OnO7ftNRY |
SUCCESS, zero fresh hits, no writes |
| Declined approval | 2vdohVf4xD7kHs9T9YmvKF |
SUCCESS, no writes |
| Approved recovery | 7K2HDM4IRrKnxkgGbKqWdD |
SUCCESS, checkpoint 18 to 19, one fresh hit |
| Repeat | 7ikjFo1RIIsVfNG21dwvXm |
SUCCESS, healthy and no writes |
| Disabled source | 3xFk6R7vDKeUrFoyqvToEy |
SUCCESS, source spared |
| Approval race | 5KnQlBHsTL62FICwNxAf3s |
FAILED before Toggle |
| Partial interruption | 5004v6AVbCwJrMYBHqF03m |
FAILED after Toggle and before reset |
io.kestra.plugin:plugin-quickwit:1.0.0 loaded.ts.SLACK_WEBHOOK_URL: Slack Incoming Webhook used by errors and the stale detector alert.| Input | Default | Purpose |
|---|---|---|
quickwit_url |
http://localhost:7280 |
Quickwit REST base URL |
index_id |
logs |
One target index |
source_id |
source |
One named Kafka source |
freshness_minutes |
5 |
Freshness watermark window |
wait_seconds |
20 |
Bounded post-reset wait |
auto_remediate |
false |
Enables the approval branch |
dry_run |
true |
Keeps writes disabled |
SLACK_WEBHOOK_URL.quickwit_url, index_id and source_id to the real target.auto_remediate: false and dry_run: true to inspect the detector.auto_remediate: true, dry_run: false,
approve the Pause and wait for the recovery proof.Add a per-index freshness budget only when the index timestamp semantics are equivalent to
ts and keep each source reset behind its own approval. The flow does not prove Kafka broker
health or guarantee that every historical document is replayed before the wait expires.
Limits: a source whose checkpoint is already advancing but whose documents are malformed can still fail the freshness check; the flow reports that failure and leaves the source enabled.
This blueprint was created by zkasuran.