Plivo SMS Delivery Failure Sentinel for Kestra

id: plivo-sms-delivery-failure-sentinel
namespace: company.team
description: |
  Every hour, count recent outbound SMS in the Plivo Message API by final
  state (read-only: no message is sent) and classify SMS delivery as
  HEALTHY, WARNING, CRITICAL, LOW_VOLUME or UNREACHABLE. Log quiet runs and
  alert Slack when messages start failing or going undelivered, with the
  most common Plivo error code.

inputs:
  - id: lookback_hours
    type: INT
    defaults: 2
    displayName: "Lookback Window (hours)"
    description: "Judge outbound SMS whose message_time is in the last this many hours."
  - id: api_timezone
    type: STRING
    defaults: "UTC"
    displayName: "Plivo Filter Time Zone"
    description: "Time zone used to render the message_time__gt filter. Plivo
      returns message_time with your account offset (for example +05:30); set
      Asia/Kolkata if counts look shifted by that offset."
  - id: warning_failure_rate
    type: FLOAT
    defaults: 3.0
    displayName: "Warning Failure Rate (%)"
    description: "At or above this share of failed plus undelivered SMS among
      finished ones, the check is WARNING."
  - id: critical_failure_rate
    type: FLOAT
    defaults: 10.0
    displayName: "Critical Failure Rate (%)"
    description: "At or above this share of failed plus undelivered SMS among
      finished ones, the check is CRITICAL."
  - id: min_messages
    type: INT
    defaults: 50
    displayName: "Minimum Messages"
    description: "Below this many finished SMS, the rate is too noisy to judge and
      the run is LOW_VOLUME."

triggers:
  - id: hourly_check
    type: io.kestra.plugin.core.trigger.Schedule
    description: Check SMS delivery every hour. Shipped disabled; enable it once the
      Plivo secrets are set and a manual run has been verified.
    cron: "10 * * * *"
    timezone: Asia/Kolkata
    disabled: true

tasks:
  - id: count_delivered
    type: io.kestra.plugin.core.http.Request
    description: Count outbound SMS in the window that reached the handset
      (message_state=delivered). Only meta.total_count is used. This is a GET;
      no message is sent.
    uri: "https://api.plivo.com/v1/Account/{{ secret('PLIVO_AUTH_ID')
      }}/Message/?message_direction=outbound&message_type=sms&message_time__gt=\
      {{ now() | dateAdd(-inputs.lookback_hours, 'HOURS') | date('yyyy-MM-dd
      HH:mm:ss', timeZone=inputs.api_timezone) | urlencode
      }}&message_state=delivered&limit=1"
    method: GET
    options:
      auth:
        type: BASIC
        username: "{{ secret('PLIVO_AUTH_ID') }}"
        password: "{{ secret('PLIVO_AUTH_TOKEN') }}"
      allowFailed: true
      timeout:
        connectTimeout: PT10S
        readIdleTimeout: PT60S
    retry:
      type: constant
      interval: PT10S
      maxAttempts: 3
    allowFailure: true

  - id: list_failed
    type: io.kestra.plugin.core.http.Request
    description: Count outbound SMS in the window that Plivo or the carrier rejected
      (message_state=failed), and read up to 20 of them for their error codes.
      This is a GET; no message is sent.
    uri: "https://api.plivo.com/v1/Account/{{ secret('PLIVO_AUTH_ID')
      }}/Message/?message_direction=outbound&message_type=sms&message_time__gt=\
      {{ now() | dateAdd(-inputs.lookback_hours, 'HOURS') | date('yyyy-MM-dd
      HH:mm:ss', timeZone=inputs.api_timezone) | urlencode
      }}&message_state=failed&limit=20"
    method: GET
    options:
      auth:
        type: BASIC
        username: "{{ secret('PLIVO_AUTH_ID') }}"
        password: "{{ secret('PLIVO_AUTH_TOKEN') }}"
      allowFailed: true
      timeout:
        connectTimeout: PT10S
        readIdleTimeout: PT60S
    retry:
      type: constant
      interval: PT10S
      maxAttempts: 3
    allowFailure: true

  - id: list_undelivered
    type: io.kestra.plugin.core.http.Request
    description: Count outbound SMS in the window that the carrier accepted but
      could not deliver (message_state=undelivered), and read up to 20 of them
      for their error codes. This is a GET; no message is sent.
    uri: "https://api.plivo.com/v1/Account/{{ secret('PLIVO_AUTH_ID')
      }}/Message/?message_direction=outbound&message_type=sms&message_time__gt=\
      {{ now() | dateAdd(-inputs.lookback_hours, 'HOURS') | date('yyyy-MM-dd
      HH:mm:ss', timeZone=inputs.api_timezone) | urlencode
      }}&message_state=undelivered&limit=20"
    method: GET
    options:
      auth:
        type: BASIC
        username: "{{ secret('PLIVO_AUTH_ID') }}"
        password: "{{ secret('PLIVO_AUTH_TOKEN') }}"
      allowFailed: true
      timeout:
        connectTimeout: PT10S
        readIdleTimeout: PT60S
    retry:
      type: constant
      interval: PT10S
      maxAttempts: 3
    allowFailure: true

  - id: summarize_messages
    type: io.kestra.plugin.core.debug.Return
    description: Add up delivered, failed and undelivered SMS from meta.total_count,
      compute the failure share, find the most common error code among the
      sampled failures, and classify in one jq query. Any missing or non-200
      response becomes UNREACHABLE.
    format: >-
      {{ (outputs.count_delivered.code is not defined or
      outputs.count_delivered.code != 200 or outputs.list_failed.code is not
      defined or outputs.list_failed.code != 200 or
      outputs.list_undelivered.code is not defined or
      outputs.list_undelivered.code != 200) ?
      '{"status":"UNREACHABLE","finished":0,"delivered":0,"failed":0,"undelivered":0,"rate":0,"top_error":"none","top_error_count":0}'
      : (fromJson('{"d":' ~ outputs.count_delivered.body ~ ',"f":' ~
      outputs.list_failed.body ~ ',"u":' ~ outputs.list_undelivered.body ~ '}')
      | jq(' (.d.meta.total_count // 0) as $ok | (.f.meta.total_count // 0) as
      $failed | (.u.meta.total_count // 0) as $undelivered | ($ok + $failed +
      $undelivered) as $n | ($failed + $undelivered) as $bad | (if $n == 0 then
      0 else (($bad * 1000 / $n) | floor) / 10 end) as $r | ([(.f.objects //
      [])[], (.u.objects // [])[]] | map((.error_code // "unknown") | tostring |
      split("\"") | join("") | .[0:40]) | group_by(.) | map({code: .[0], count:
      length}) | sort_by(-.count)) as $errs | {finished: $n, delivered: $ok,
      failed: $failed, undelivered: $undelivered, rate: $r, top_error: (if
      ($errs | length) > 0 then $errs[0].code else "none" end), top_error_count:
      (if ($errs | length) > 0 then $errs[0].count else 0 end), status: (if $n <
      ' ~ inputs.min_messages ~ ' then "LOW_VOLUME" elif $r >= ' ~
      inputs.critical_failure_rate ~ ' then "CRITICAL" elif $r >= ' ~
      inputs.warning_failure_rate ~ ' then "WARNING" else "HEALTHY" end)} ') |
      first | toJson) }}

  - id: route_by_failures
    type: io.kestra.plugin.core.flow.Switch
    description: Route to a quiet log when delivery is healthy or volume is too low
      to judge, a Slack message when failures cross the warning line, or a Slack
      page when they cross the critical line or Plivo could not be read.
    value: "{{ fromJson(outputs.summarize_messages.value).status }}"
    cases:
      "HEALTHY":
        - id: log_healthy
          type: io.kestra.plugin.core.log.Log
          description: Log a quiet line when SMS delivery is healthy; no Slack message is
            sent.
          message: "Plivo SMS healthy: {{
            fromJson(outputs.summarize_messages.value).failed }} failed and {{
            fromJson(outputs.summarize_messages.value).undelivered }}
            undelivered of {{
            fromJson(outputs.summarize_messages.value).finished }} finished SMS
            ({{ fromJson(outputs.summarize_messages.value).rate }}%) in the last
            {{ inputs.lookback_hours }} hour(s)."
      "LOW_VOLUME":
        - id: log_low_volume
          type: io.kestra.plugin.core.log.Log
          description: Log a quiet line when there is too little traffic in the window to
            judge; no Slack message is sent.
          message: "Plivo SMS: only {{ fromJson(outputs.summarize_messages.value).finished
            }} finished outbound SMS in the last {{ inputs.lookback_hours }}
            hour(s), below the minimum of {{ inputs.min_messages }}; rate not
            judged this run."
      "WARNING":
        - id: alert_warning_sms
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          description: Post a Slack message when the failure rate is at or above the
            warning line.
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {
              "text": "Plivo SMS WARNING: {{ fromJson(outputs.summarize_messages.value).failed }} failed and {{ fromJson(outputs.summarize_messages.value).undelivered }} undelivered of {{ fromJson(outputs.summarize_messages.value).finished }} finished SMS ({{ fromJson(outputs.summarize_messages.value).rate }}%) in the last {{ inputs.lookback_hours }} hour(s). Most common error code: {{ fromJson(outputs.summarize_messages.value).top_error }} (x{{ fromJson(outputs.summarize_messages.value).top_error_count }} in the sample). Execution {{ execution.id }}."
            }
      "CRITICAL":
        - id: alert_critical_sms
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          description: Page Slack when the failure rate is at or above the critical line,
            which usually means a sender ID, a DLT template, the account balance
            or a carrier route is broken.
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {
              "text": "Plivo SMS CRITICAL: {{ fromJson(outputs.summarize_messages.value).failed }} failed and {{ fromJson(outputs.summarize_messages.value).undelivered }} undelivered of {{ fromJson(outputs.summarize_messages.value).finished }} finished SMS ({{ fromJson(outputs.summarize_messages.value).rate }}%) in the last {{ inputs.lookback_hours }} hour(s). Most common error code: {{ fromJson(outputs.summarize_messages.value).top_error }} (x{{ fromJson(outputs.summarize_messages.value).top_error_count }} in the sample). OTPs and alerts are not reaching customers; check the sender ID, templates, balance and the Plivo error code reference now. Execution {{ execution.id }}."
            }
      "UNREACHABLE":
        - id: alert_unreachable_api
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          description: Page Slack when the Message API could not be read, indicating an
            outage or invalid credentials.
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {
              "text": "Plivo SMS sentinel UNREACHABLE: the Message API did not return a usable response (HTTP delivered={{ outputs.count_delivered.code ?? 'none' }}, failed={{ outputs.list_failed.code ?? 'none' }}, undelivered={{ outputs.list_undelivered.code ?? 'none' }}). Check the auth ID and token. Execution {{ execution.id }}."
            }
    defaults:
      - id: log_unclassified
        type: io.kestra.plugin.core.log.Log
        description: Fallback for any classification the cases above do not cover, so an
          unexpected value is visible in the logs.
        message: "Plivo SMS sentinel completed with an unexpected classification: {{
          outputs.summarize_messages.value }}."

  - id: log_audit_result
    type: io.kestra.plugin.core.log.Log
    description: Record one audit line for every run, whichever branch fired, so the
      delivery history stays queryable even if Slack delivery fails.
    message: "Plivo SMS sentinel audit: lookback_hours={{ inputs.lookback_hours }}
      http={{ outputs.count_delivered.code ?? 'n/a' }}/{{
      outputs.list_failed.code ?? 'n/a' }}/{{ outputs.list_undelivered.code ??
      'n/a' }} summary={{ outputs.summarize_messages.value }} execution={{
      execution.id }}."

errors:
  - id: alert_flow_failure
    type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
    description: Alert Slack when the sentinel flow itself fails, so a broken check
      is never mistaken for healthy delivery.
    url: "{{ secret('SLACK_WEBHOOK') }}"
    payload: |
      {
        "text": "Plivo SMS sentinel flow {{ flow.id }} FAILED (execution {{ execution.id }}). SMS delivery health is now unknown; check the task logs and the Plivo message logs."
      }

outputs:
  - id: status
    type: STRING
    value: "{{ fromJson(outputs.summarize_messages.value).status }}"

OTPs, order updates and payment alerts sent through Plivo fail quietly: a sender ID or DLT template gets rejected, the account balance runs out, a number is blocked by a carrier, or a route has trouble. Each message just ends failed or undelivered with an error_code, and the first signal is usually customers saying they never got the OTP. Plivo's delivery callbacks report one message at a time; nothing watches the rate. This blueprint turns three scheduled, read-only calls to the Plivo Message API into a delivery health gate: io.kestra.plugin.core.http.Request counts delivered, failed and undelivered SMS, one jq query computes the failure rate and the most common error code, and the result routes to a quiet log or a Slack alert.

How it works

count_delivered, list_failed and list_undelivered (io.kestra.plugin.core.http.Request) call GET /v1/Account/{auth_id}/Message/ with message_direction=outbound, message_type=sms, a message_time__gt filter for the lookback window and one message_state each, using HTTP basic auth with the auth ID and token. Each response carries meta.total_count, so three small calls give exact counts without paging; the failed and undelivered calls also read up to 20 messages for their error_code. They are reads: no message is sent. Retries cover short network blips, and allowFailed plus allowFailure keep an API error from aborting the flow so it can be reported instead.
summarize_messages (io.kestra.plugin.core.debug.Return) first guards for any missing or non-200 response (UNREACHABLE). Otherwise one jq query adds the three counts, computes the share that failed or went undelivered, and finds the most common error_code in the sample. Messages still queued or sent are not counted, because their outcome is not known yet. Fewer than min_messages finished SMS is LOW_VOLUME; at or above critical_failure_rate is CRITICAL; at or above warning_failure_rate is WARNING; otherwise HEALTHY.
route_by_failures (io.kestra.plugin.core.flow.Switch) branches on that classification: HEALTHY and LOW_VOLUME log quietly, WARNING posts a Slack note, CRITICAL and UNREACHABLE page Slack. A defaults fallback logs anything unexpected.
log_audit_result (io.kestra.plugin.core.log.Log) writes one audit line per run with the three HTTP statuses and the full summary.
The flow-level errors block pages Slack if the flow itself fails, and the hourly_check trigger runs at 10 minutes past every hour (Asia/Kolkata), shipped disabled: true.

What you get

An hourly, read-only check of SMS delivery with no custom scripting and no extra plugin to install.
A status output (HEALTHY / WARNING / CRITICAL / LOW_VOLUME / UNREACHABLE) other flows can gate on, for example to switch OTPs to WhatsApp or voice while SMS fails.
Slack messages with failed and undelivered counts, the failure rate and the most common Plivo error code.
An audit line on every run, separate from Slack.

Who it's for

Teams sending OTPs, transactional alerts or order updates through Plivo, including Indian senders bound by DLT templates.
Support and growth teams that need to know before customers do when messages stop arriving.
Teams that already run their alerting in Kestra.

Why orchestrate this with Kestra

Plivo's console shows message logs and its callbacks report one message at a time. Kestra turns plain, read-only API calls into a scheduled, typed classification with retries, a failure path and an execution history, and the status output lets your notification flows fall back to another channel automatically.

Prerequisites

A Plivo account sending SMS, with its Auth ID and Auth Token from the Plivo console overview page.
A Slack incoming webhook for the WARNING, CRITICAL, UNREACHABLE and flow-failure alerts.

Secrets

PLIVO_AUTH_ID: the Auth ID (basic auth username and part of the URL path).
PLIVO_AUTH_TOKEN: the Auth Token (basic auth password).
SLACK_WEBHOOK: Slack incoming webhook used for every alert branch and the flow-failure alert.

In Kestra OSS, secrets are supplied as environment variables prefixed SECRET_ and base64-encoded, for example SECRET_PLIVO_AUTH_ID=$(echo -n 'MAXXXXXXXXXXXXXXXXXX' | base64), and read back with {{ secret('PLIVO_AUTH_ID') }}. Use the Enterprise secrets backend for encryption at rest and access control.

Quick start

Send a few test SMS through the Plivo API (or note a busy hour of real traffic).
Add PLIVO_AUTH_ID, PLIVO_AUTH_TOKEN and SLACK_WEBHOOK as secrets.
Run the flow manually with a low min_messages and compare the counts in the logs with the message logs in the Plivo console; set api_timezone to Asia/Kolkata if they are off by your account's offset.
Enable hourly_check.

How to extend

Add a subaccount or powerpack_id filter to watch one product or sender pool with its own limits.
Change message_type to whatsapp or mms for a second sentinel on those channels.
Restrict the trigger cron to your sending hours, and store the last status in a KV store to alert only on changes.

Inputs

lookback_hours: type INT, default 2.
api_timezone: type STRING, default UTC.
warning_failure_rate: type FLOAT, default 3.0 (percent).
critical_failure_rate: type FLOAT, default 10.0 (percent).
min_messages: type INT, default 50.

Outputs

status: one of HEALTHY, WARNING, CRITICAL, LOW_VOLUME, UNREACHABLE.
Example path: {{ fromJson(outputs.summarize_messages.value).top_error }} for the most common error code in the sample.

Pitfalls

The error code is taken from a sample of up to 20 failed and 20 undelivered messages (the API page limit); the counts themselves are exact.
queued and sent messages are left out until they reach a final state, so a very short window can look healthier than it is.
Check the message_time__gt time zone against your account (see api_timezone).
This flow only reads; it never sends, cancels or deletes a message.