Schedule icon
GetHealth icon
Switch icon
Log icon
SlackIncomingWebhook icon

Ceph Cluster Health Gate

Fetch Ceph's own cluster health report via the Dashboard API and alert Slack on HEALTH_WARN or HEALTH_ERR, read-only.

Categories
DataInfrastructure

Ceph already knows its own health - ceph health or the Dashboard's health panel reports HEALTH_OK, HEALTH_WARN, or HEALTH_ERR the moment something changes. What it does not do on its own is page anyone. This blueprint calls the Ceph Dashboard's own GET /api/health/full endpoint on a schedule, routes directly on the cluster's own three-state status, and alerts Slack with the exact per-check summary messages the Dashboard itself produced - not a derived or re-interpreted signal.

How it works

  1. periodic_health_check (io.kestra.plugin.core.trigger.Schedule) runs every 15 minutes. Shipped disabled so you can validate a manual run first.
  2. check_cluster_health (io.kestra.plugin.ceph.cluster.GetHealth) calls GET /api/health/full, returning status (HEALTH_OK/HEALTH_WARN/HEALTH_ERR), summary (a list of human-readable per-check messages), and checks (the raw per-check detail objects).
  3. evaluate_cluster_health (io.kestra.plugin.core.flow.Switch on status) routes to one of three cases, plus a defensive default.
    • "HEALTH_OK": log_cluster_healthy records a clean check.
    • "HEALTH_WARN": alert_cluster_warn posts the Dashboard's own summary messages to Slack.
    • "HEALTH_ERR": alert_cluster_err posts the same summary with more urgent wording, since this is Ceph's most severe state.
    • default: log_unexpected_status logs any status string outside those three, in case a future Ceph version adds one.
  4. log_audit_complete always runs last, printing the headline status regardless of which branch fired.
  5. The errors block alerts Slack separately if the flow itself fails outright - an unreachable Dashboard host or bad credentials must not read as "no risk".

What you get

  • A direct read of Ceph's own three-state cluster health, not a derived or threshold-based reinterpretation of lower-level metrics.
  • The Dashboard's own human-readable summary messages in the Slack alert, so the on-call engineer sees what actually tripped, not just a bare status string.
  • A defensive default branch so an unrecognized future status value is logged distinctly instead of silently matching no case.
  • A read-only design: this flow calls exactly one GET endpoint - it cannot create, modify, or delete a pool, RBD image, or any other cluster object.

Who it's for

  • Storage and platform teams running Ceph who want cluster health pushed to Slack instead of relying on someone checking the Dashboard or running ceph health by hand.
  • SREs who want the exact Ceph-generated check messages in the alert itself, so triage starts with real detail instead of a bare "warning" notification.
  • Anyone evaluating the Ceph plugin who wants a worked example of cluster.GetHealth, a plugin with zero prior coverage in this repository despite real, substantial task classes across cluster, pools, and rbd.

Why orchestrate this with Kestra

ceph health run by hand, or the Dashboard's own panel, answers one moment in time; it does not decide a cadence, route on the status value as a first-class branch, or notify anyone. Kestra supplies the schedule, the three-way routing with a defensive fallback, and an execution history that shows exactly when a cluster moved between HEALTH_OK, HEALTH_WARN, and HEALTH_ERR.

Prerequisites

  • A reachable Ceph cluster with the Dashboard Manager module enabled and its REST API reachable from the Kestra Worker.
  • A Dashboard account (or a pre-obtained JWT for a one-off run - see Pitfalls for why that is unsafe on a schedule) with permission to read /api/health/full.
  • A Slack incoming webhook for alerts.

Local testing: Ceph is a multi-daemon distributed system (monitors, managers, OSDs, and the Dashboard module all run as separate processes) - there is no official single-container demo image for the Dashboard API, confirmed against Ceph's own containerized-deployment documentation, which only lists production (quay.io/ceph/ceph) and development (quay.ceph.io/ceph-ci/ceph) images with no demo/quickstart variant. Use Ceph's own cephadm bootstrap quickstart (which stands up a single-node cluster with the Dashboard enabled in Docker/Podman containers) rather than a hand-assembled single container; once it is running, confirm the Dashboard's actual host and port before setting ceph_host/ceph_port, and avoid host port 8080 if Kestra's own UI already uses it on the same machine - the plugin's own default Dashboard port is 8443, which does not collide.

Secrets

  • CEPH_USERNAME / CEPH_PASSWORD: Dashboard account credentials, exchanged by the plugin for a session token via POST /api/auth on every task run.
  • SLACK_WEBHOOK_URL: Slack incoming webhook used by alert_cluster_warn, alert_cluster_err, and the errors block.
  • In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded, and read back in flows with {{ secret('NAME') }} - for example SECRET_CEPH_PASSWORD=$(echo -n '<your-local-password>' | base64). This keeps credentials out of the flow YAML but, per Kestra's own documentation, offers no encryption at rest or access control beyond the host environment; use the Enterprise secrets backend for stronger guarantees.

Inputs

  • ceph_host (STRING, default localhost): Ceph Dashboard hostname or IP.
  • ceph_port (INT, default 8443): Ceph Dashboard REST API port.

Outputs

  • outputs.check_cluster_health.status / .summary / .checks: the full health report, every run.

Quick start

  1. Stand up a Ceph cluster with the Dashboard module enabled, for example via cephadm bootstrap, and note its Dashboard host and port.
  2. Add the CEPH_USERNAME, CEPH_PASSWORD, and SLACK_WEBHOOK_URL secrets.
  3. Run the flow manually with the defaults and confirm log_cluster_healthy fires on a healthy cluster.
  4. Induce a warning (take an OSD down, or let a pool approach its near-full ratio) and re-run to confirm alert_cluster_warn fires with the Dashboard's own summary text.
  5. Enable periodic_health_check once you trust the check, using username/password credentials rather than a pre-obtained token.

How to extend

  • Add io.kestra.plugin.ceph.pools.List/Get as a second probe to break a HEALTH_WARN caused by a near-full pool into its own, more specific Slack message naming the exact pool and its quota usage.
  • Route HEALTH_ERR to a page instead of a Slack message, since it is Ceph's own most severe state and likely means reduced data redundancy or availability right now.
  • Replace the Schedule trigger with io.kestra.plugin.ceph.cluster.HealthTrigger, a real-time trigger exposed by this same plugin, if event-driven response time matters more than a polling cadence.
  • Parse outputs.check_cluster_health.checks (the raw per-check objects, not just the summary strings) to branch on a specific check code, such as treating a POOL_NEAR_FULL check differently from a OSD_DOWN check even though both currently only produce the same generic HEALTH_WARN alert.

Pitfalls

  • A pre-obtained token is unsafe on a schedule. Verified from the plugin's own source: a token is used as-is for the whole execution, has Ceph's default 8-hour TTL, and is never auto-renewed by this plugin - the scheduled trigger must use username/password, which are exchanged for a fresh token on every run, not a hardcoded token.
  • HEALTH_WARN and HEALTH_ERR both only alert - neither remediates. Verified from the plugin's own task list: cluster only exposes GetHealth/GetStatus/HealthTrigger, all read-only. Fixing a warning or error needs a cause-specific action (adding capacity, restarting an OSD, adjusting a pool's near-full ratio) this flow cannot infer from a status string alone.
  • The default Switch case is not decorative. Ceph's documented states are HEALTH_OK, HEALTH_WARN, HEALTH_ERR, but a Switch with no matching case and no default silently does nothing - log_unexpected_status exists specifically so a future or unanticipated status value is still visible in the execution log.
  • skipSsl defaults to false, but most Ceph Dashboards use a self-signed certificate. Verified from AbstractCephConnection's own documentation: this is the documented default behavior for Ceph, not a misconfiguration - set skipSsl: true on check_cluster_health explicitly if your Dashboard has not been issued a trusted certificate, understanding that this disables TLS verification.
  • port defaults to 8443 on the plugin itself, not 8080. This flow's own ceph_port input mirrors that default; there is no port conflict with Kestra's own UI to configure around here, unlike blueprints whose target service defaults to 8080.
  • This blueprint is UNTESTED against a live Ceph cluster. GetHealth's properties, its HealthOutput fields (status, summary, checks), and AbstractCephConnection's connection fields (host, port, username, password, token, skipSsl) are taken verbatim from the plugin's source on main - but no Docker/Ceph environment or Kestra engine was run to execute this flow end to end.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.