Couchbase N1QL Compliance Violation Sentinel

A business rule encoded only in application code, a record that should never stay pending past an hour, a document that should never be missing a required field, can still slip through if a bug or a manual data fix bypasses that code path. Couchbase itself has no built-in rule-check that pages anyone the moment such a document appears; it just sits there until a report looks wrong. This blueprint turns a single N1QL query encoding that rule into a standing, real-time compliance check.

This blueprint was created by Mohit Jeswani.

How it works

1. watch_violations (io.kestra.plugin.couchbase.Trigger) polls query every 5 minutes and only fires an execution when it returns at least one row; a clean poll produces no execution at all, so every execution of this flow represents a real, currently-existing violation.
2. evaluate (io.kestra.plugin.core.output.OutputValues) classifies the violation count this poll found as WIDESPREAD_VIOLATION at or above max_isolated_violation_count, otherwise VIOLATION_DETECTED, and collects the violating document ids with a jq filter.
3. route (io.kestra.plugin.core.flow.If) posts to Slack either way, since both severities represent a real violation, with the widespread case worded for urgency and carrying the same real document ids either way.
4. summary logs the violation count and severity, closing the governance cycle.
5. errors alerts Slack if the flow itself fails, for example a broker connection drop.

What you get

• A violation check built on Couchbase's own N1QL query engine, not a periodic report someone has to remember to read.
• The real violating document ids (via META().id) in the Slack alert, not just a count, so whoever responds can act without a separate lookup.
• A trigger that produces zero noise when the data is clean: no execution, no log entry, nothing, until an actual violation exists.

Who it's for

• Data platform and backend teams running Couchbase who encode a business rule in a N1QL query and want it enforced as a standing check, not a one-off script run during an incident.
• Teams evaluating the Couchbase plugin who want a worked example of the real-time Trigger used for data-compliance enforcement, distinct from this repo's existing Couchbase blueprints, which all use the one-shot Query task for index, cluster, and TTL governance rather than a continuously-polling violation trigger.

Why orchestrate this with Kestra

Couchbase answers a N1QL query on request; it does not keep polling on its own, decide what counts as a widespread versus isolated violation, or notify a Slack channel the moment one appears. Kestra keeps max_isolated_violation_count as a flow input, turns a violating query result into a Slack page carrying the real document ids, and keeps a full execution history of every violation ever caught.

Prerequisites

• A Couchbase cluster, locally testable via the official couchbase/server Docker image, with a bucket, scope, and collection your query can check a business rule against.
• A Couchbase user with N1QL query access to that collection.
• A Slack incoming webhook URL.

Secrets

• COUCHBASE_USERNAME / COUCHBASE_PASSWORD: used by watch_violations. In Kestra OSS, provide them as the base64-encoded environment variables SECRET_COUCHBASE_USERNAME and SECRET_COUCHBASE_PASSWORD.
• SLACK_WEBHOOK_URL: used by both Slack tasks and the errors block. In Kestra OSS, provide it as the base64-encoded environment variable SECRET_SLACK_WEBHOOK_URL.

Inputs

Input	Default	Purpose
max_isolated_violation_count	3	Row count at or above which a poll is classified as widespread rather than isolated

Outputs

• {{ trigger.rows }} / .size: the violating documents this poll found and how many, as the Trigger reports them (reusing the underlying Query task's own output shape).
• {{ outputs.evaluate.values.violation_count }} / .violation_ids: the same count, and the violating document ids, semicolon-joined.
• {{ outputs.evaluate.values.severity }}: VIOLATION_DETECTED or WIDESPREAD_VIOLATION.

Pitfalls

• query is the entire governance policy; this blueprint does not validate that it actually encodes a real rule, a loosely-written N1QL query can match far more (or fewer) documents than intended.
• The example query projects META().id AS doc_id; a query that renames or omits that alias breaks the jq filter that collects document ids for the Slack alert, which looks for that exact key.
• fetchType: FETCH keeps rows in memory for the Slack payload; a query that can legitimately match a very large number of documents should lower fetchType to STORE and alert with the storage URI instead of inlining every row into one Slack message.

Quick start

1. Start a local Couchbase cluster (docker run -p 8091-8096:8091-8096 -p 11210:11210 couchbase/server:community) and add the COUCHBASE_USERNAME, COUCHBASE_PASSWORD, and SLACK_WEBHOOK_URL secrets.
2. Set connectionString, query on watch_violations to a real bucket/scope/collection and a real business rule.
3. Insert a document that violates the rule and enable the trigger; confirm an alert fires with that document's real id.
4. Insert enough violating documents at once to confirm WIDESPREAD_VIOLATION fires instead of the isolated case.

How to extend

• Chain a SELECT COUNT(*) FROM ... query via io.kestra.plugin.couchbase.Query in the widespread branch to size the full scope of the violation beyond this one poll's rows.
• Add a remediation branch that updates the violating documents to a quarantine flag via io.kestra.plugin.couchbase.Query, gated behind an explicit confirm_quarantine input.
• Route WIDESPREAD_VIOLATION to an on-call paging channel and the isolated case to a lower-urgency channel, since the two likely have different owners.

Links

• Couchbase plugin: https://kestra.io/plugins/plugin-couchbase
• Couchbase Docker image: https://hub.docker.com/r/couchbase/server
• Slack incoming webhooks: https://api.slack.com/messaging/webhooks