Schedule icon
Request icon
Switch icon
SlackIncomingWebhook icon
Fail icon
Log icon

OpenBao Seal Status and Cluster Health Sentinel

Audit OpenBao secret management cluster health, detect sealed instances, verify Raft storage readiness, and dispatch Slack alerts on authentication outages.

Categories
Infrastructure

OpenBao is the Linux Foundation open-source, community-governed identity-based secret management and encryption platform. Widely deployed across cloud-native environments, OpenBao manages database credentials, API tokens, cryptographic certificates, and transit encryption keys for modern container workloads and automated delivery pipelines.

In production operations, OpenBao instances initialize in a locked, encrypted state known as Sealed. Following unexpected node reboots, container restarts, or Raft storage quorum failovers, an unsealed node automatically re-seals if auto-unseal mechanisms fail. While sealed, all secret access operations fail immediately, causing cascading outages across applications that depend on dynamic secret retrieval. This blueprint implements an automated Sentinel that audits OpenBao health, detects sealed states in real time, and alerts on-call teams via Slack before application failures propagate.

How it works

  1. Scheduled Audit Trigger: Runs on a configurable 15-minute interval (*/15 * * * *), disabled by default for controlled onboarding and ad-hoc manual validation.
  2. Health Probe Request: Dispatches an HTTP GET request to /v1/sys/health with allowFailed: true and configurable standbyok parameters, capturing status codes and JSON cluster metadata.
  3. State Classification: Evaluates HTTP response codes and payload attributes within a flow-control Switch task, routing executions into OPENBAO_SEALED (HTTP 503), UNINITIALIZED (HTTP 501), ENDPOINT_UNREACHABLE, or HEALTHY (HTTP 200/429) pathways.
  4. Slack Incident Alerting: Issues structured incident notifications via io.kestra.plugin.slack.notifications.SlackIncomingWebhook containing target host coordinates, HTTP status, and operational unseal runbooks.
  5. Deterministic Failure & Audit: Terminates degraded executions with io.kestra.plugin.core.execution.Fail to enforce SLA tracking, and emits a final audit log capturing execution telemetry.

What you get

  • Instant Seal Detection: Immediate alerting when an OpenBao server node reboots into a sealed state, preventing prolonged secret downtime.
  • Initialization & Unreachable Tracking: Detects uninitialized cluster members or unresponsive daemon processes across ingress boundaries.
  • Triage-Ready Slack Notifications: Formatted alerts delivering exact host coordinates, HTTP response codes, and actionable unseal instructions.
  • Declarative Security Observability: Replaces fragile shell scripts and custom sidecars with auditable, version-controlled YAML.

Who it's for

  • Security & DevSecOps Engineers: Managing centralized secret management clusters, dynamic credentials, and PKI engines.
  • Platform & Infrastructure Teams: Responsible for secret store availability, Raft storage quorum, and zero-trust authentication pipelines.
  • Site Reliability Engineers: Monitoring critical path infrastructure dependencies across production Kubernetes clusters.

Why orchestrate this with Kestra

  • Resilient Error Handling: Uses allowFailed: true to inspect non-200 HTTP responses without prematurely terminating the workflow.
  • Stateful Flow Branching: Classifies complex JSON responses with Pebble expressions without requiring bespoke monitoring microservices.
  • Dynamic Secret Protection: Protects Slack webhook URLs and administrative tokens using native namespace secret management.
  • Auditable History: Maintains an immutable execution log of every health check for SOC 2 and security compliance reviews.

Prerequisites

  • A running OpenBao server instance with network accessibility from the Kestra worker.
  • Network connectivity to the OpenBao API port (8200).
  • An incoming webhook URL configured in your target Slack workspace.

Secrets

  • SLACK_WEBHOOK_URL: Target Slack incoming webhook URL for operational alerts and incident escalation (required).

Quick start

  1. Configure the SLACK_WEBHOOK_URL secret in your Kestra namespace settings.
  2. Start a local OpenBao container in dev mode for testing: docker run -d --name openbao -p 8200:8200 -e 'BAO_DEV_ROOT_TOKEN_ID=root' openbao/openbao:latest server -dev -dev-listen-address=0.0.0.0:8200.
  3. Import this blueprint into Kestra and trigger an execution with default inputs.
  4. Confirm that the health probe succeeds and verify the nominal status in the Kestra UI execution logs.
  5. Enable the schedule trigger to establish automated 15-minute continuous monitoring.

How to extend

  • Automated Unseal Trigger: Add tasks to invoke cloud KMS or unseal key orchestration when a sealed node is detected.
  • Raft Peer Verification: Chain additional HTTP requests to /v1/sys/storage/raft/configuration to verify cluster peer voting health.
  • PagerDuty Escalation: Add io.kestra.plugin.pagerduty.notifications.PagerDutyAlert for high-severity sealed outage events.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.