AI Model Health and Availability Watchdog with Kestra

id: ai-model-health-availability-watchdog
namespace: company.team

description: |
  Production-oriented AI model health and availability watchdog.

  This workflow checks configured AI models for:
  - API availability
  - real inference health
  - inference latency against a configurable SLO

  Each model is processed independently with controlled concurrency.
  The workflow first checks whether the configured model endpoint is
  available and then performs a real inference smoke test when available.

inputs:
  - id: models
    type: ARRAY
    itemType: STRING
    defaults:
      - '{"name":"Groq GPT-OSS
        20B","model":"openai/gpt-oss-20b","base_url":"https://api.groq.com/openai/v1","api_key_secret":"GROQ_API_KEY"}'
    description: |
      JSON objects describing the AI models to monitor.

      Each object must contain:
      - name
      - model
      - base_url
      - api_key_secret

  - id: latency_slo
    type: DURATION
    defaults: PT10S
    description: |
      Maximum allowed duration for the inference smoke test.

  - id: smoke_prompt
    type: STRING
    defaults: "Reply with exactly: HEALTHY"
    description: |
      Minimal prompt used to verify that the model can perform
      an actual inference request.

tasks:
  - id: check_models
    type: io.kestra.plugin.core.flow.Loop
    values: "{{ inputs.models }}"
    concurrencyLimit: 3
    fetchType: FETCH
    outputs:
      - id: status
        type: STRING
        value: "{{ outputs.model_status.values.status }}"
    tasks:
      - id: model_lookup
        type: io.kestra.plugin.core.http.Request
        uri: "{{ fromJson(item.value).base_url }}/models/{{ fromJson(item.value).model
          }}"
        method: GET
        options:
          allowFailed: true
        headers:
          Authorization: "Bearer {{ secret(fromJson(item.value).api_key_secret) }}"
          Content-Type: application/json

      - id: smoke_test
        type: io.kestra.plugin.ai.completion.ChatCompletion
        runIf: "{{ outputs.model_lookup.code == 200 }}"
        allowFailure: true
        timeout: "{{ inputs.latency_slo }}"
        provider:
          type: io.kestra.plugin.ai.provider.OpenAI
          apiKey: "{{ secret(fromJson(item.value).api_key_secret) }}"
          modelName: "{{ fromJson(item.value).model }}"
          baseUrl: "{{ fromJson(item.value).base_url }}"
        messages:
          - type: USER
            content: "{{ inputs.smoke_prompt }}"

      - id: model_result
        type: io.kestra.plugin.core.log.Log
        message: |
          ==========================================
          AI MODEL HEALTH CHECK
          ==========================================

          Model:
          {{ fromJson(item.value).name }}

          Model ID:
          {{ fromJson(item.value).model }}

          Availability HTTP Code:
          {{ outputs.model_lookup.code }}

          Inference State:
          {{ tasks.smoke_test.state }}

          Inference Duration:
          {{ outputs.smoke_test.requestDuration | default("N/A") }} ms

          Status:
          {{
            outputs.model_lookup.code != 200
            ? "UNAVAILABLE"
            : (
                tasks.smoke_test.state == "SUCCESS"
                ? "HEALTHY"
                : "UNHEALTHY"
              )
          }}

          ==========================================

      - id: model_status
        type: io.kestra.plugin.core.output.OutputValues
        values:
          status: >-
            {{
              outputs.model_lookup.code != 200
              ? "UNAVAILABLE"
              : (
                  tasks.smoke_test.state == "SUCCESS"
                  ? "HEALTHY"
                  : "UNHEALTHY"
                )
            }}

  - id: health_report
    type: io.kestra.plugin.core.log.Log
    message: |
      ==================================================
              AI MODEL HEALTH REPORT
      ==================================================

      Configured models:
      {{ inputs.models | length }}

      Model statuses:
      {{ loopOutputs(outputs.check_models.outputs, 'status') }}

      ==================================================

  - id: health_decision
    type: io.kestra.plugin.core.flow.If
    condition: >-
      {{
        (loopOutputs(outputs.check_models.outputs, 'status') contains "UNAVAILABLE")
        or
        (loopOutputs(outputs.check_models.outputs, 'status') contains "UNHEALTHY")
      }}
    then:
      - id: degraded_status
        type: io.kestra.plugin.core.log.Log
        message: |
          ⚠ AI MODEL HEALTH STATUS: DEGRADED

          One or more configured models are unavailable
          or failed the inference health check.

          Review the model health report above.
    else:
      - id: healthy_status
        type: io.kestra.plugin.core.log.Log
        message: |
          ✓ AI MODEL HEALTH STATUS: HEALTHY

          All configured models passed availability
          and inference health checks.

outputs:
  - id: monitoring_status
    type: STRING
    value: >-
      {{
        (loopOutputs(outputs.check_models.outputs, 'status') contains "UNAVAILABLE")
        or
        (loopOutputs(outputs.check_models.outputs, 'status') contains "UNHEALTHY")
        ? "DEGRADED"
        : "HEALTHY"
      }}

  - id: models_checked
    type: INT
    value: "{{ inputs.models | length }}"

triggers:
  - id: scheduled_health_check
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 */6 * * *"
    timezone: Asia/Kolkata
    disabled: true

AI model dependencies can fail in ways that are difficult to detect before they affect an application. An endpoint may be reachable while a specific model is unavailable, or a model may respond successfully but exceed an acceptable latency target.

This Blueprint provides a reusable health watchdog that checks configured AI models on a schedule, verifies model availability through the provider API, performs a real inference smoke test, and records the resulting health state.

Models are processed with controlled concurrency so multiple configured providers can be monitored without overwhelming the workflow or provider.

Features

Check configured model endpoints for availability.
Perform real inference smoke tests when a model is available.
Apply a configurable inference timeout using latency_slo.
Process multiple configured models with controlled concurrency.
Classify models as HEALTHY, UNAVAILABLE, or UNHEALTHY.
Aggregate model results into an overall HEALTHY or DEGRADED status.
Support scheduled monitoring, with the schedule disabled by default.

Prerequisites

A running Kestra instance with the required HTTP and AI plugins.
A valid provider API key stored as a Kestra secret.
A model endpoint accessible from the Kestra instance.

Quick start

Import this flow into Kestra.
Configure the models input with the model name, model ID, base URL, and API-key secret name.
Configure latency_slo and smoke_prompt as required.
Execute the flow and inspect the model health report.
Enable the scheduled trigger if recurring monitoring is required.

Expected results

HEALTHY: the model endpoint is available and inference succeeds.
UNAVAILABLE: the model endpoint does not return HTTP 200.
UNHEALTHY: the endpoint is available but the inference check fails.
DEGRADED: at least one configured model is unavailable or unhealthy.
HEALTHY overall: all configured models pass their checks.

Notes

The workflow uses the configured provider API and performs a real inference request. The scheduled trigger is disabled by default. Configure a valid secret and confirm the required plugins are installed before execution.