Trigger icon
OutputValues icon
If icon
SlackIncomingWebhook icon
Log icon

ScyllaDB Compliance Violation Flag Sentinel

Poll ScyllaDB for compliance violations, flag and alert on each one once, and log the result for data platform teams.

Categories
Data

A row that silently violates a business rule, a record stuck pending past its deadline, a document missing a required field, is easy to miss one row at a time and easy to re-alert on forever if the same check runs again before anyone fixes it. ScyllaDB's own CQL has no built-in "alert me once per violation" primitive. This blueprint uses the plugin's acknowledge-on-read trigger to flag each violating row the instant it alerts on it, so the exact same row is never paged twice.

This blueprint was created by Mohit Jeswani.

How it works

  1. watch_violations (io.kestra.plugin.scylladb.Trigger) runs cql every minute, selecting rows that violate the rule and have not been flagged yet. Each returned row is then acknowledged with acknowledgeCql, an idempotent UPDATE that sets flagged_at, in the same poll, before the flow's own tasks run. A poll with no matching rows fires no execution at all.
  2. evaluate (io.kestra.plugin.core.output.OutputValues) classifies the violation count this poll found as WIDESPREAD_VIOLATION at or above max_isolated_violation_count, otherwise VIOLATION_DETECTED, and collects the now-flagged row ids with a jq filter.
  3. route (io.kestra.plugin.core.flow.If) posts to Slack either way, since both severities represent a real violation, with the widespread case worded for urgency.
  4. summary logs the count and severity, closing the governance cycle.
  5. errors alerts Slack if the flow itself fails, for example a ScyllaDB connection drop.

What you get

  • A violation check that never pages twice on the same row, since flagging happens as part of the trigger's own poll, not as a separate task that could be skipped.
  • The exact violating row ids in the Slack alert, so whoever responds can jump straight to the record and clear the flag after remediation.
  • A trigger that produces zero noise when the data is clean: no execution, no log entry, nothing, until an actual, not-yet-flagged violation exists.

Who it's for

  • Data platform and backend teams running ScyllaDB who encode a business rule in CQL and want it enforced as a standing, deduplicated check, not a one-off script someone reruns and double-counts stale alerts from.
  • Teams evaluating the ScyllaDB plugin who want a worked example of the acknowledge-on-read Trigger used for compliance enforcement, distinct from this repo's existing ScyllaDB blueprints, which all use the one-shot Query task for cluster, partition, and schema governance rather than a continuously-polling, self-deduplicating violation trigger.

Why orchestrate this with Kestra

ScyllaDB answers a CQL query on request; it does not keep polling on its own, decide what counts as a widespread versus isolated violation, flag what it already reported, or notify a Slack channel when one appears. Kestra keeps max_isolated_violation_count as a flow input, turns a violating query result into a Slack page carrying the real, now-flagged row ids, and keeps a full execution history of every violation ever caught.

Prerequisites

  • A ScyllaDB cluster, locally testable via the official scylladb/scylla Docker image, with a table that has a nullable flagged_at timestamp column alongside the fields your rule checks.
  • A ScyllaDB user with read and write access to that table, since acknowledgement requires an UPDATE.
  • A Slack incoming webhook URL.

Secrets

  • SCYLLADB_HOST / SCYLLADB_USERNAME / SCYLLADB_PASSWORD: used by watch_violations. In Kestra OSS, provide them as the base64-encoded environment variables SECRET_SCYLLADB_HOST, SECRET_SCYLLADB_USERNAME, and SECRET_SCYLLADB_PASSWORD.
  • SLACK_WEBHOOK_URL: used by both Slack tasks and the errors block. In Kestra OSS, provide it as the base64-encoded environment variable SECRET_SLACK_WEBHOOK_URL.

Inputs

Input Default Purpose
max_isolated_violation_count 3 Row count at or above which a poll is classified as widespread rather than isolated

Outputs

  • {{ trigger.rows }} / .size: the violating, now-flagged rows this poll found and how many, as the Trigger reports them (reusing the underlying Query task's own output shape).
  • {{ outputs.evaluate.values.violation_count }} / .violation_ids: the same count, and the flagged row ids, semicolon-joined.
  • {{ outputs.evaluate.values.severity }}: VIOLATION_DETECTED or WIDESPREAD_VIOLATION.

Pitfalls

  • acknowledgeCql is required by this trigger and must be an idempotent UPDATE or DELETE binding the row's full primary key; this blueprint uses UPDATE ... SET flagged_at = ... specifically so the violating row is marked reviewed, not destroyed.
  • This flow is not purely read-only: setting flagged_at is a real write, deliberately scoped to exactly the rows just alerted about, as the dedup mechanism itself.
  • cql's flagged_at IS NULL clause is what prevents double-alerting; clearing that column manually after remediation (or on a schedule) is required for a row to ever be eligible for this check again if the same condition recurs.

Quick start

  1. Start a local ScyllaDB cluster (docker run -p 9042:9042 scylladb/scylla) and add the three ScyllaDB secrets plus SLACK_WEBHOOK_URL.
  2. Add a nullable flagged_at timestamp column to a real table, set keyspace, cql, and acknowledgeCql on watch_violations to match it.
  3. Insert a row that violates the rule and enable the trigger; confirm an alert fires with its real id and that flagged_at is now set on that row.
  4. Insert enough violating rows at once to confirm WIDESPREAD_VIOLATION fires instead of the isolated case.

How to extend

  • Add a separate scheduled flow that clears flagged_at on rows where the underlying violation has since been resolved, so a recurring issue on the same row can alert again.
  • Chain io.kestra.plugin.scylladb.Query in the widespread branch to run an aggregate CQL count across the whole table, not just this one poll's LIMIT-bounded batch.
  • Route WIDESPREAD_VIOLATION to an on-call paging channel and the isolated case to a lower-urgency channel, since the two likely have different owners.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.