ScyllaDB Large Partition Sentinel

A partition that grows past ScyllaDB's own compaction-time size or row-count threshold gets written to system.large_partitions, ScyllaDB's own standing record of exactly which keyspace, table, and partition key crossed the line. Nothing in ScyllaDB itself alerts on new entries in that table, so a large partition, a well known cause of hot spots, slow compactions, and timeout cascades, can sit there for weeks before anyone notices. This blueprint reads that table on a schedule, classifies the cluster from what is actually recorded, and alerts Slack naming the real offender, never a guess.

How it works

1. probe_large_partitions (io.kestra.plugin.scylladb.Query) runs SELECT keyspace_name, table_name, sstable_name, partition_size, rows, compaction_time FROM system.large_partitions, with allowFailure: true so a bad contact point or credential does not stop the flow before it can be classified as PROBE_FAILED below.
2. classify_large_partition_status (io.kestra.plugin.core.debug.Return) returns PROBE_FAILED when the probe did not return at all, HEALTHY when the table is empty, CRITICAL when the largest recorded partition_size is at or above critical_partition_size_mb (converted to bytes for the comparison), WARNING otherwise.
3. route_by_large_partition_status (io.kestra.plugin.core.flow.Switch) branches on that status: HEALTHY logs it, the other three each alert Slack, the WARNING and CRITICAL alerts name the real keyspace_name, table_name, and partition_size of the single largest recorded partition, never an invented value. An unexpected value falls through to a defaults branch that logs it rather than being silently dropped.
4. log_audit_summary always runs last and logs the final classification.
5. The errors block posts to Slack if the flow itself fails outside the gate logic above.

What you get

• A real read of ScyllaDB's own large-partition log, not a guess from cluster load or latency graphs.
• A Slack alert naming the actual keyspace, table, and byte size responsible, never an invented value.
• A second, configurable severity tier above ScyllaDB's own built-in detection threshold, so routine, already-known-about large partitions can stay at WARNING while a genuinely oversized one pages as CRITICAL.
• A fully local test path against the official ScyllaDB Docker image, no managed cluster needed.

Who it's for

• Database reliability and platform teams running ScyllaDB who want to know about a newly-recorded large partition without manually querying system.large_partitions.
• Teams evaluating the ScyllaDB plugin who want a working example of io.kestra.plugin.scylladb.Query against a real system table, with native CQL authentication.

Why orchestrate this with Kestra

ScyllaDB records large partitions, it does not notify anyone about them, that gap usually gets filled by a cron job and a hand-rolled alerting script. Kestra replaces that script with a declarative flow: the schedule, the severity classification against your own second threshold, and the Slack alert naming the real offender are all one auditable execution history.

Prerequisites

• For local testing, the official ScyllaDB Docker image:

  docker run --name scylla -d -p 9042:9042 scylladb/scylla:latest --smp 1
  

  --smp 1 limits ScyllaDB to one shard, recommended for a laptop-sized container. Wait for docker exec -it scylla nodetool status to report UN (up, normal) before connecting, ScyllaDB takes a little while to finish starting.

• Authentication is disabled by default on a fresh ScyllaDB container. To test with username/password set, enable authenticator: PasswordAuthenticator in scylla.yaml and restart the container, then connect as the default cassandra / cassandra superuser account to create your own role.

• To actually populate system.large_partitions for testing, lower compaction_large_partition_warning_threshold_mb well below its 1000MB default in scylla.yaml, then write and compact a partition past that lowered threshold.

• A Slack incoming webhook for alerts.

Secrets

• SCYLLADB_USERNAME, SCYLLADB_PASSWORD: native CQL authentication credentials. Only required when the cluster has PasswordAuthenticator enabled.

• SLACK_WEBHOOK: Slack incoming webhook URL used by the alert tasks and the errors block.

  In Kestra OSS, set these as base64-encoded environment variables, for example SECRET_SCYLLADB_USERNAME, SECRET_SLACK_WEBHOOK.

Inputs

Input	Type	Default	Allowed values
contact_points	STRING	localhost:9042	comma-separated host:port list
local_datacenter	STRING	datacenter1	any datacenter name known to the driver
critical_partition_size_mb	INT	2048	any positive integer, in megabytes

Quick start

1. Start the ScyllaDB container above.
2. Lower compaction_large_partition_warning_threshold_mb and write a partition past it, per Prerequisites, so system.large_partitions has at least one row.
3. Add the SLACK_WEBHOOK secret, and the ScyllaDB credentials if authentication is enabled.
4. Run this flow manually with the defaults, it should report WARNING or CRITICAL naming the real keyspace, table, and size you just created.
5. Enable the every_30_minutes trigger.

How to extend

• Add a second Query against system.large_rows or system.large_cells, ScyllaDB's equivalent logs for an oversized row or cell within an otherwise normal-sized partition.
• Loop this flow with io.kestra.plugin.core.flow.Loop across contact_points per cluster if you operate more than one ScyllaDB cluster from one Kestra instance.
• Join the alert against your own schema-ownership mapping to route the Slack message to the team that owns the specific keyspace found, instead of a single shared channel.

Pitfalls

• system.large_partitions is ScyllaDB's own standing log, this blueprint does not clear or TTL it. A repeated WARNING/CRITICAL across runs can mean the same historical entry is still being read, not that a new one just appeared. Manage this table's retention per ScyllaDB's own operational guidance if that distinction matters to you.
• Every row already exceeded ScyllaDB's own compaction_large_partition_warning_threshold_mb (1000MB by default) before it was ever written to this table, critical_partition_size_mb is a second bar on top of that, not the only line of defense.
• This gate only reads and alerts, there is no CQL-level fix for an oversized partition, resolving one needs a data modeling change, which needs a human decision, not an automated task.
• A fresh ScyllaDB container defaults to AllowAllAuthenticator, username/password are simply ignored until PasswordAuthenticator is explicitly enabled.

Links

• ScyllaDB plugin documentation
• ScyllaDB Docker image
• ScyllaDB large partitions table