Detect Qdrant embedding drift and verify bounded re-embedding
Compare live embeddings with stored vectors and repair an approved batch only when held-out retrieval has degraded.
Poll ScyllaDB for compliance violations, flag and alert on each one once, and log the result for data platform teams.
A row that silently violates a business rule, a record stuck pending past its deadline, a
document missing a required field, is easy to miss one row at a time and easy to re-alert
on forever if the same check runs again before anyone fixes it. ScyllaDB's own CQL has no
built-in "alert me once per violation" primitive. This blueprint uses the plugin's
acknowledge-on-read trigger to flag each violating row the instant it alerts on it, so the
exact same row is never paged twice.
This blueprint was created by Mohit Jeswani.
watch_violations (io.kestra.plugin.scylladb.Trigger) runs cql every minute,
selecting rows that violate the rule and have not been flagged yet. Each returned row is
then acknowledged with acknowledgeCql, an idempotent UPDATE that sets flagged_at,
in the same poll, before the flow's own tasks run. A poll with no matching rows fires no
execution at all.evaluate (io.kestra.plugin.core.output.OutputValues) classifies the violation count
this poll found as WIDESPREAD_VIOLATION at or above max_isolated_violation_count,
otherwise VIOLATION_DETECTED, and collects the now-flagged row ids with a jq filter.route (io.kestra.plugin.core.flow.If) posts to Slack either way, since both
severities represent a real violation, with the widespread case worded for urgency.summary logs the count and severity, closing the governance cycle.errors alerts Slack if the flow itself fails, for example a ScyllaDB connection drop.Trigger used for compliance enforcement, distinct from this repo's existing ScyllaDB
blueprints, which all use the one-shot Query task for cluster, partition, and schema
governance rather than a continuously-polling, self-deduplicating violation trigger.ScyllaDB answers a CQL query on request; it does not keep polling on its own, decide what
counts as a widespread versus isolated violation, flag what it already reported, or notify
a Slack channel when one appears. Kestra keeps max_isolated_violation_count as a flow
input, turns a violating query result into a Slack page carrying the real, now-flagged row
ids, and keeps a full execution history of every violation ever caught.
scylladb/scylla Docker image, with
a table that has a nullable flagged_at timestamp column alongside the fields your rule
checks.UPDATE.SCYLLADB_HOST / SCYLLADB_USERNAME / SCYLLADB_PASSWORD: used by watch_violations.
In Kestra OSS, provide them as the base64-encoded environment variables
SECRET_SCYLLADB_HOST, SECRET_SCYLLADB_USERNAME, and SECRET_SCYLLADB_PASSWORD.SLACK_WEBHOOK_URL: used by both Slack tasks and the errors block. In Kestra OSS,
provide it as the base64-encoded environment variable SECRET_SLACK_WEBHOOK_URL.| Input | Default | Purpose |
|---|---|---|
max_isolated_violation_count |
3 |
Row count at or above which a poll is classified as widespread rather than isolated |
{{ trigger.rows }} / .size: the violating, now-flagged rows this poll found and how
many, as the Trigger reports them (reusing the underlying Query task's own output
shape).{{ outputs.evaluate.values.violation_count }} / .violation_ids: the same count, and
the flagged row ids, semicolon-joined.{{ outputs.evaluate.values.severity }}: VIOLATION_DETECTED or
WIDESPREAD_VIOLATION.acknowledgeCql is required by this trigger and must be an idempotent UPDATE or
DELETE binding the row's full primary key; this blueprint uses UPDATE ... SET flagged_at = ... specifically so the violating row is marked reviewed, not destroyed.flagged_at is a real write, deliberately
scoped to exactly the rows just alerted about, as the dedup mechanism itself.cql's flagged_at IS NULL clause is what prevents double-alerting; clearing that column
manually after remediation (or on a schedule) is required for a row to ever be eligible
for this check again if the same condition recurs.docker run -p 9042:9042 scylladb/scylla) and add the
three ScyllaDB secrets plus SLACK_WEBHOOK_URL.flagged_at timestamp column to a real table, set keyspace, cql, and
acknowledgeCql on watch_violations to match it.flagged_at is now set on that row.WIDESPREAD_VIOLATION fires instead of
the isolated case.flagged_at on rows where the underlying
violation has since been resolved, so a recurring issue on the same row can alert again.io.kestra.plugin.scylladb.Query in the widespread branch to run an aggregate CQL
count across the whole table, not just this one poll's LIMIT-bounded batch.WIDESPREAD_VIOLATION to an on-call paging channel and the isolated case to a
lower-urgency channel, since the two likely have different owners.