List icon
Return icon
Switch icon
Log icon
SlackIncomingWebhook icon
Schedule icon

DigitalOcean Alert Policy Compliance Gate

List DigitalOcean monitoring alert policies on a schedule and alert Slack naming the real ones that are disabled.

Categories
CloudInfrastructure

A DigitalOcean monitoring alert policy that gets disabled during an incident, a maintenance window, or a debugging session does not announce itself, the resource it watches keeps running, nothing in the DigitalOcean UI flags it as a gap, and the policy simply stops firing. This blueprint lists every alert policy on the account on a schedule, counts the real ones that are disabled, and alerts Slack naming exactly which ones, never an invented count.

How it works

  1. probe_alert_policies (io.kestra.plugin.digitalocean.monitoring.alertpolicy.List) lists every alert policy on the account, with allowFailure: true so a bad or expired API token does not stop the flow before it can be classified as PROBE_FAILED below.
  2. classify_alert_policy_status (io.kestra.plugin.core.debug.Return) returns PROBE_FAILED when the probe did not return at all, CRITICAL or WARNING from the real count of policies where enabled is false, against the two thresholds, HEALTHY otherwise.
  3. route_by_alert_policy_status (io.kestra.plugin.core.flow.Switch) branches on that status: HEALTHY logs it, the other three alert Slack, WARNING and CRITICAL name the real disabled policy types and UUIDs, never an invented list. A defaults branch logs anything unexpected.
  4. log_audit_summary always runs last. The errors block posts to Slack if the flow itself fails outside the gate logic above.

What you get

  • A real list of exactly which alert policies are disabled, not a guess from whether anyone has complained about a missed alert yet.
  • A Slack alert naming the real policy type and UUID, never a generic notice.
  • A tiered severity you control, separate from DigitalOcean's own enabled flag.

Who it's for

  • Platform and SRE teams running infrastructure on DigitalOcean who need to know when monitoring coverage quietly regresses, not after an incident nothing paged on.
  • Teams evaluating the DigitalOcean plugin who want a working example of monitoring.alertpolicy.List and its pagination.

Why orchestrate this with Kestra

DigitalOcean has no built-in alert for "an alert policy itself is disabled", that is exactly the kind of meta-monitoring gap a dashboard will not catch since a disabled policy produces no signal at all. Kestra adds the schedule, the threshold policy, and the Slack alert naming the real policies as one declarative, auditable flow.

Prerequisites

  • A DigitalOcean account and a personal access token with read access to monitoring, generated under API in the DigitalOcean control panel. DigitalOcean's monitoring API has no official local emulator, this gate runs against a real account; a free-tier account and token are sufficient, no paid plan is required to call this endpoint.
  • A Slack incoming webhook for alerts.

Secrets

  • DIGITALOCEAN_TOKEN: personal access token used by the probe task.

  • SLACK_WEBHOOK: Slack incoming webhook URL used by the alert tasks and the errors block.

    In Kestra OSS, set these as base64-encoded environment variables, for example SECRET_DIGITALOCEAN_TOKEN, SECRET_SLACK_WEBHOOK.

Inputs

  • disabled_count_warning_threshold (INT, default 1): Number of disabled alert policies at or above which the account is classified WARNING.
  • disabled_count_critical_threshold (INT, default 3): Number of disabled alert policies at or above which the account is classified CRITICAL.

Outputs

  • {{ outputs.probe_alert_policies.total }}: Total number of alert policies discovered on the account.
  • {{ outputs.probe_alert_policies.rows }}: List of alert policy maps containing uuid, type, and enabled boolean flags.
  • {{ outputs.classify_alert_policy_status.value }}: The resulting compliance classification (HEALTHY, WARNING, CRITICAL, or PROBE_FAILED).

Quick start

  1. Generate a DigitalOcean API token and add the two secrets above.
  2. Disable one existing alert policy, or create one and disable it, to have at least one disabled policy to see.
  3. Run this flow manually with the defaults, it should report WARNING naming the policy you just disabled.
  4. Re-enable it, run again, it should report HEALTHY.
  5. Enable the every_morning trigger.

How to extend

  • Add a check for policies with an empty alerts.slack and alerts.email list, an enabled policy with no notification channel configured is as silent as a disabled one.
  • Cross-reference entities/tags against your own list of production droplet IDs or tags to flag a production resource with no alert policy covering it at all, not only an existing policy that got disabled.
  • Loop this flow with io.kestra.plugin.core.flow.Loop per DigitalOcean team if you manage several teams' accounts from one Kestra instance.

Pitfalls

  • This gate only reads and alerts, it never re-enables a disabled policy automatically, someone disabled it for a reason that needs a human to confirm before flipping it back on.
  • enabled reflects the policy's own setting, not whether its notification channel (Slack, email, PagerDuty) is itself working, a policy can be enabled and still not reach anyone if its webhook URL is stale.
  • DigitalOcean's API returns up to 200 policies per page; probe_alert_policies follows pagination automatically, but an account with an extremely large number of policies will make more than one API call per run.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.