Orthanc DICOM Transfer Job Sentinel for Kestra

id: orthanc-dicom-transfer-job-sentinel
namespace: company.team
description: |
  On a schedule, read-only check the jobs of an Orthanc DICOM server (nothing
  is resent, cancelled or deleted) and classify image routing as HEALTHY,
  WARNING, CRITICAL or UNREACHABLE. Log quiet runs and page Slack when studies
  fail to reach the PACS, a CAD or triage service, or an archive, or when transfers are
  stuck, so a scan never silently misses the radiologist's worklist.

inputs:
  - id: orthanc_url
    type: STRING
    defaults: "https://orthanc.example.org"
    displayName: "Orthanc URL"
    description: "Base URL of the Orthanc REST API, without a trailing slash."
  - id: lookback_minutes
    type: INT
    defaults: 60
    displayName: "Lookback (minutes)"
    description: "Failed jobs created within this window are counted. Match it to
      the schedule interval plus a margin."
  - id: stuck_minutes
    type: INT
    defaults: 30
    displayName: "Stuck After (minutes)"
    description: "A job still Pending, Running, Paused or Retry this long after it
      was created counts as stuck."
  - id: critical_failed
    type: INT
    defaults: 3
    displayName: "Critical Failed Jobs"
    description: "At or above this many failed jobs in the lookback window, the
      check is classified CRITICAL."
  - id: critical_stuck_minutes
    type: INT
    defaults: 120
    displayName: "Critical Stuck (minutes)"
    description: "If any job has been stuck this long, the check is classified CRITICAL."

triggers:
  - id: routing_check
    type: io.kestra.plugin.core.trigger.Schedule
    description: Check image routing every 15 minutes. Shipped disabled; enable it
      once orthanc_url and the ORTHANC_USERNAME and ORTHANC_PASSWORD secrets
      point at a real server and a manual run has been verified.
    cron: "*/15 * * * *"
    disabled: true
    inputs:
      orthanc_url: "https://orthanc.example.org"

tasks:
  - id: read_system
    type: io.kestra.plugin.core.http.Request
    description: Read /system to confirm the endpoint is a reachable Orthanc server
      and record its version. This is a GET; nothing is changed.
    uri: "{{ inputs.orthanc_url }}/system"
    method: GET
    options:
      auth:
        type: BASIC
        username: "{{ secret('ORTHANC_USERNAME') }}"
        password: "{{ secret('ORTHANC_PASSWORD') }}"
      allowFailed: true
      timeout:
        connectTimeout: PT10S
        readIdleTimeout: PT30S
    retry:
      type: constant
      interval: PT15S
      maxAttempts: 3
    allowFailure: true

  - id: fetch_jobs
    type: io.kestra.plugin.core.http.Request
    description: Read every job Orthanc still remembers with /jobs?expand, including
      its state, type, creation time, error and the destination (RemoteAet) of
      DICOM transfers. Skipped when /system did not answer like Orthanc.
    runIf: "{{ outputs.read_system.code is defined and outputs.read_system.code ==
      200 and (outputs.read_system.body | trim | startsWith('{')) }}"
    uri: "{{ inputs.orthanc_url }}/jobs?expand"
    method: GET
    options:
      auth:
        type: BASIC
        username: "{{ secret('ORTHANC_USERNAME') }}"
        password: "{{ secret('ORTHANC_PASSWORD') }}"
      allowFailed: true
      timeout:
        connectTimeout: PT10S
        readIdleTimeout: PT60S
    retry:
      type: constant
      interval: PT15S
      maxAttempts: 3
    allowFailure: true

  - id: classify_routing
    type: io.kestra.plugin.core.debug.Return
    description: Classify the jobs in one jq query. Orthanc job times are UTC in the
      compact form 20261016T091653.093748, so they are compared as strings with
      thresholds in the same form. A missing response, a non-200 code, a body
      that is not JSON, or a /system answer without an Orthanc version become
      UNREACHABLE instead of looking healthy.
    format: >-
      {{ (outputs.read_system.code is not defined or outputs.read_system.code !=
      200 or not (outputs.read_system.body | trim | startsWith('{')) or
      outputs.fetch_jobs is not defined or outputs.fetch_jobs.code is not
      defined or outputs.fetch_jobs.code != 200 or not (outputs.fetch_jobs.body
      | trim | startsWith('['))) ?
      '{"status":"UNREACHABLE","jobs":0,"failed":0,"stuck":0,"failing_destinations":"none","top_error":"none","oldest_stuck":"none"}'
      : ({'s': fromJson(outputs.read_system.body), 'j':
      fromJson(outputs.fetch_jobs.body)} | jq(' "' ~ (now() |
      dateAdd(-inputs.lookback_minutes, 'MINUTES') | date("yyyyMMdd'T'HHmmss",
      timeZone='UTC')) ~ '" as $since | "' ~ (now() |
      dateAdd(-inputs.stuck_minutes, 'MINUTES') | date("yyyyMMdd'T'HHmmss",
      timeZone='UTC')) ~ '" as $stuck_before | "' ~ (now() |
      dateAdd(-inputs.critical_stuck_minutes, 'MINUTES') |
      date("yyyyMMdd'T'HHmmss", timeZone='UTC')) ~ '" as $critical_before | if
      (.s.Version // "") == "" then {status: "UNREACHABLE", jobs: 0, failed: 0,
      stuck: 0, failing_destinations: "none", top_error: "none", oldest_stuck:
      "none"} else [.j[] | select(.State == "Failure" and .CreationTime >=
      $since)] as $failed | [.j[] | select((.State == "Pending" or .State ==
      "Running" or .State == "Paused" or .State == "Retry") and .CreationTime <
      $stuck_before)] as $stuck | {orthanc_version: .s.Version, jobs: (.j |
      length), failed: ($failed | length), stuck: ($stuck | length),
      failing_destinations: (if ($failed | length) == 0 then "none" else
      ([$failed[] | (.Content.RemoteAet // .Type)] | group_by(.) | map(.[0] + "
      (" + (length | tostring) + ")") | join(", ")) end), top_error: (if
      ($failed | length) == 0 then "none" else ([$failed[] | (.ErrorDescription
      // "unknown")] | group_by(.) | max_by(length) | .[0]) end), oldest_stuck:
      (if ($stuck | length) == 0 then "none" else ($stuck |
      min_by(.CreationTime) | .Type + " " + .State + " since " +
      .CreationTime[0:15] + " UTC (job " + .ID + ")") end), status: (if ($failed
      | length) >= ' ~ inputs.critical_failed ~ ' or ([$stuck[] |
      select(.CreationTime < $critical_before)] | length) > 0 then "CRITICAL"
      elif ($failed | length) > 0 or ($stuck | length) > 0 then "WARNING" else
      "HEALTHY" end)} end ') | first | toJson) }}

  - id: route_by_routing_status
    type: io.kestra.plugin.core.flow.Switch
    description: Route to a quiet log when no transfer failed or is stuck, an
      informational Slack message for a few failures or a stuck job, or a Slack
      page for repeated failures, a long-stuck job, or an unreachable server.
    value: "{{ fromJson(outputs.classify_routing.value).status }}"
    cases:
      "HEALTHY":
        - id: log_healthy
          type: io.kestra.plugin.core.log.Log
          description: Log a quiet confirmation when no job failed in the lookback window
            and none is stuck.
          message: "Orthanc routing healthy: {{
            fromJson(outputs.classify_routing.value).jobs }} job(s) known, none
            failed in the last {{ inputs.lookback_minutes }} minute(s), none
            stuck."
      "WARNING":
        - id: alert_warning_routing
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          description: Post an informational Slack message with the failing destinations,
            the most common error and the oldest stuck job.
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {
              "text": "Orthanc routing WARNING: {{ fromJson(outputs.classify_routing.value).failed }} failed job(s) in the last {{ inputs.lookback_minutes }} minute(s), {{ fromJson(outputs.classify_routing.value).stuck }} stuck. Failing destinations: {{ fromJson(outputs.classify_routing.value).failing_destinations }}. Most common error: {{ fromJson(outputs.classify_routing.value).top_error }}. Oldest stuck: {{ fromJson(outputs.classify_routing.value).oldest_stuck }}. Execution {{ execution.id }}."
            }
      "CRITICAL":
        - id: alert_critical_routing
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          description: Page Slack when transfers keep failing or a job has been stuck past
            critical_stuck_minutes; studies may not be reaching the PACS or the
            radiologist.
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {
              "text": "Orthanc routing CRITICAL: {{ fromJson(outputs.classify_routing.value).failed }} failed job(s) in the last {{ inputs.lookback_minutes }} minute(s), {{ fromJson(outputs.classify_routing.value).stuck }} stuck. Failing destinations: {{ fromJson(outputs.classify_routing.value).failing_destinations }}. Most common error: {{ fromJson(outputs.classify_routing.value).top_error }}. Oldest stuck: {{ fromJson(outputs.classify_routing.value).oldest_stuck }}. Studies may not be reaching their destination. Execution {{ execution.id }}."
            }
      "UNREACHABLE":
        - id: alert_unreachable_orthanc
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          description: Page Slack when Orthanc could not be read or did not answer like
            Orthanc.
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {
              "text": "Orthanc routing UNREACHABLE: {{ inputs.orthanc_url }} did not return its system information and job list (server down, wrong URL, rejected credentials, or a proxy page instead of Orthanc). Image routing status is unknown. Execution {{ execution.id }}."
            }
    defaults:
      - id: log_unclassified
        type: io.kestra.plugin.core.log.Log
        description: Log any classification outside the four known values, so a change
          in the jq query never fails silently.
        message: "Orthanc DICOM transfer job sentinel completed with an unexpected
          classification: {{ outputs.classify_routing.value }}."

  - id: log_audit_result
    type: io.kestra.plugin.core.log.Log
    description: Record one audit line for every run, whichever branch fired, with
      both HTTP codes and the classification, so the check's own history stays
      queryable even if Slack delivery fails.
    message: "Orthanc routing audit: server={{ inputs.orthanc_url }} system_http={{
      outputs.read_system.code ?? 'n/a' }} jobs_http={{ outputs.fetch_jobs.code
      ?? 'n/a' }} result={{ outputs.classify_routing.value }} execution={{
      execution.id }}."

errors:
  - id: alert_flow_failure
    type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
    description: Alert Slack when the sentinel flow itself fails outright, so a
      broken check is never mistaken for healthy image routing.
    url: "{{ secret('SLACK_WEBHOOK') }}"
    payload: |
      {
        "text": "Orthanc DICOM transfer job sentinel flow {{ flow.id }} FAILED (execution {{ execution.id }}). Image routing at {{ inputs.orthanc_url }} is unknown; check the task logs and the Orthanc jobs page manually."
      }

outputs:
  - id: status
    type: STRING
    value: "{{ outputs.classify_routing is defined ?
      fromJson(outputs.classify_routing.value).status : 'UNREACHABLE' }}"
  - id: routing
    type: JSON
    value: "{{ outputs.classify_routing is defined ? outputs.classify_routing.value
      : '{\"status\":\"UNREACHABLE\"}' }}"

Imaging departments use Orthanc as a router: scans arrive from the modalities and Orthanc forwards them to the PACS, a CAD or triage service, a teleradiology partner or an archive. When a destination is down, rejects the association, or a network link drops, the transfer job fails or sits in Retry, and nothing pages anyone. The study is simply missing from the radiologist's worklist until someone asks where it is. This blueprint turns two read-only Orthanc REST calls into a scheduled image routing gate: /system proves the server is answering, /jobs?expand returns every job with its state, destination and error, a single jq query counts failed and stuck transfers, and the result routes to a quiet log, an informational Slack note or a Slack page.

How it works

read_system (io.kestra.plugin.core.http.Request) reads /system with basic authentication. Only an answer with an Orthanc Version counts as a working server. Every request is a GET with a timeout, three retries, allowFailed: true and allowFailure: true, so an outage never aborts the flow.
fetch_jobs reads /jobs?expand: every job Orthanc still remembers, with State (Pending, Running, Success, Failure, Paused, Retry), Type, CreationTime, ErrorDescription and, for DICOM transfers, the destination in Content.RemoteAet. A runIf skips it when /system did not answer.
classify_routing (io.kestra.plugin.core.debug.Return) checks both responses (a missing response, a non-200 code or a body that is not JSON is UNREACHABLE), then counts in one jq query the Failure jobs created in the last lookback_minutes and the jobs still Pending, Running, Paused or Retry after stuck_minutes. It names the failing destinations with their counts, the most common error, and the oldest stuck job. At least critical_failed failures, or a job stuck past critical_stuck_minutes, is CRITICAL; any failure or stuck job is WARNING; otherwise HEALTHY.
route_by_routing_status (io.kestra.plugin.core.flow.Switch) logs HEALTHY quietly, posts a Slack note for WARNING, pages Slack for CRITICAL and UNREACHABLE, and logs any unexpected value in defaults.
log_audit_result writes one audit line per run with both HTTP codes and the classification. The flow-level errors block alerts Slack if the flow fails outright.
The routing_check trigger runs every 15 minutes and ships disabled: true.

What you get

A scheduled, read-only routing check for any Orthanc server, using only its REST API.
A status output (HEALTHY / WARNING / CRITICAL / UNREACHABLE) other flows can gate on, and a routing JSON output with jobs, failed, stuck, failing_destinations, top_error, oldest_stuck and orthanc_version.
Slack messages that name the destination and the error, for example Failing destinations: PACS (3). Most common error: Error in the network protocol.
An audit log line on every run, so routing history survives a Slack outage.

Who it's for

Radiology and imaging IT teams that route studies through Orthanc to a PACS, CAD or triage services, teleradiology or archives.
Hospitals and clinics running Orthanc as a mini-PACS or research archive.
Data platform teams already using Kestra who want the same alerting for their imaging infrastructure.

Why orchestrate this with Kestra

Orthanc records every job and its error, but only someone looking at the jobs page sees them. Kestra runs the same read-only check on a schedule, classifies all routing in one typed status, keeps every run in the execution history, and routes it to the right channel, while the Switch branches are ready to grow into follow-up (a ticket for the PACS team, a C-ECHO test of the failing destination, a resend after an approval) without changing the check.

Prerequisites

An Orthanc server whose REST API is reachable from Kestra, with authentication enabled.
An Orthanc user for the monitoring flow. Raise JobsHistorySize in the Orthanc configuration (the default keeps only the last 10 finished jobs) so the lookback window really contains every failure.
A Slack incoming webhook for the WARNING, CRITICAL, UNREACHABLE and flow-failure alerts.

Secrets

ORTHANC_USERNAME and ORTHANC_PASSWORD: credentials of the monitoring user (HTTP basic authentication).
SLACK_WEBHOOK: Slack incoming webhook used for every alert branch and the flow-failure alert.

In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_ and base64-encoded, then read in flows with {{ secret('NAME') }}, for example SECRET_ORTHANC_PASSWORD=$(echo -n '<password>' | base64).

Quick start

To try it without a production server, run Orthanc locally (the orthancteam/orthanc image, or the macOS and Windows packages from orthanc.uclouvain.be) with AuthenticationEnabled, a RegisteredUsers entry, and two DicomModalities: one that answers and one that does not (for example a port nothing listens on).
Upload a few DICOM files, then send a study to each modality with POST /modalities/<name>/store and "Asynchronous": true. The transfer to the silent modality fails within seconds.
Add ORTHANC_USERNAME, ORTHANC_PASSWORD and SLACK_WEBHOOK as secrets, set orthanc_url (for example http://localhost:8042), and run the flow. The status is WARNING and Slack names the failing destination and its error.
Enable routing_check once pointed at the real server.

How to extend

Filter failing_destinations to the modalities that matter (the PACS and the triage service) and send each team its own channel.
Add a POST /modalities/<name>/echo C-ECHO step for each failing destination to tell a network problem from a rejected study.
Store status and failed in the KV store and alert only when the failure count grows.
Read /statistics in the same flow to warn when the storage disk approaches its limit.

Inputs

orthanc_url: type STRING, default "https://orthanc.example.org".
lookback_minutes: type INT, default 60.
stuck_minutes: type INT, default 30.
critical_failed: type INT, default 3.
critical_stuck_minutes: type INT, default 120.

Outputs

status: one of HEALTHY, WARNING, CRITICAL, UNREACHABLE.
routing: the full classification as JSON, for example {"orthanc_version":"1.13.0","jobs":3,"failed":1,"stuck":0,"failing_destinations":"PACS (1)","top_error":"Error in the network protocol","oldest_stuck":"none","status":"WARNING"}.

Pitfalls

Orthanc forgets finished jobs beyond JobsHistorySize (10 by default) and, unless SaveJobs is on, after a restart. A failure that has already been pushed out of the history cannot be seen; raise the history size on busy servers.
Job times are in UTC in Orthanc's compact format (20261016T091653.093748). The query compares them with Kestra's clock in UTC, so keep the Kestra host's clock synchronised.
Orthanc users in RegisteredUsers have full access. Put the monitoring account behind the authorization plugin or a read-only reverse proxy rule if you need a read-only role; this flow only ever sends GET requests.
The check only reads. It never resends, cancels or deletes jobs.