Azure Front Door Origin Response Latency Sentinel Blueprint

id: azure-frontdoor-origin-response-latency-sentinel
namespace: company.cloud

description: |
  Audits Azure Front Door origin response latency and backend health probe statuses via Azure Monitor,
  alerting site reliability engineering teams on Slack when origins degrade or fail availability SLAs.

concurrency:
  limit: 1

inputs:
  - id: azure_subscription_id
    type: STRING
    displayName: Azure Subscription ID
    defaults: 00000000-0000-0000-0000-000000000000
    description: Target Azure subscription ID hosting the Azure Front Door profile.

  - id: resource_group_name
    type: STRING
    displayName: Resource Group Name
    defaults: production-rg
    description: Name of the Azure resource group containing the Front Door profile.

  - id: frontdoor_profile_name
    type: STRING
    displayName: Front Door Profile Name
    defaults: prod-global-frontdoor
    description: Name of the Azure Front Door Standard/Premium profile to audit.

  - id: max_origin_latency_ms
    type: INT
    displayName: Max Origin Latency (ms)
    defaults: 1200
    description: Maximum allowable origin response latency in milliseconds before
      triggering performance alerts.

  - id: alert_on_latency
    type: BOOL
    displayName: Alert on Latency
    defaults: true
    description: Whether to dispatch automated Slack incident notifications when
      origin degradation is detected.

tasks:
  - id: scan_frontdoor_origin_telemetry
    type: io.kestra.plugin.scripts.python.Script
    description: Queries Azure Monitor metrics and Front Door REST APIs to measure
      origin response latency and probe health.
    containerImage: python:3.11-slim
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
    beforeCommands:
      - pip install azure-identity azure-mgmt-monitor requests
    env:
      AZURE_CLIENT_ID: "{{ secret('AZURE_CLIENT_ID') }}"
      AZURE_CLIENT_SECRET: "{{ secret('AZURE_CLIENT_SECRET') }}"
      AZURE_TENANT_ID: "{{ secret('AZURE_TENANT_ID') }}"
      AZURE_SUBSCRIPTION_ID: "{{ inputs.azure_subscription_id }}"
    script: |
      import os
      import json
      import requests
      from datetime import datetime, timedelta, timezone
      from azure.identity import ClientSecretCredential
      from azure.mgmt.monitor import MonitorManagementClient
      from kestra import Kestra

      sub_id = os.environ.get("AZURE_SUBSCRIPTION_ID", "00000000-0000-0000-0000-000000000000")
      client_id = os.environ.get("AZURE_CLIENT_ID", "")
      client_secret = os.environ.get("AZURE_CLIENT_SECRET", "")
      tenant_id = os.environ.get("AZURE_TENANT_ID", "")

      rg_name = "{{ inputs.resource_group_name }}"
      fd_profile = "{{ inputs.frontdoor_profile_name }}"
      max_latency = int("{{ inputs.max_origin_latency_ms }}")

      resource_id = f"/subscriptions/{sub_id}/resourceGroups/{rg_name}/providers/Microsoft.Cdn/profiles/{fd_profile}"

      now = datetime.now(timezone.utc)
      start_time = now - timedelta(minutes=15)
      timespan = f"{start_time.strftime('%Y-%m-%dT%H:%M:%SZ')}/{now.strftime('%Y-%m-%dT%H:%M:%SZ')}"

      origins_audited = 0
      degraded_origins = []

      try:
          if client_id and client_secret and tenant_id:
              creds = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret)
              monitor_client = MonitorManagementClient(creds, sub_id)

              # Query OriginLatency and OriginHealthPercentage metrics
              metrics_data = monitor_client.metrics.list(
                  resource_id,
                  timespan=timespan,
                  interval="PT5M",
                  metricnames="OriginLatency,OriginHealthPercentage",
                  aggregation="Average"
              )

              avg_latency = 0.0
              health_pct = 100.0

              for m in metrics_data.value:
                  if m.name.value == "OriginLatency":
                      for ts in m.timeseries:
                          for d in ts.data:
                              if d.average is not None:
                                  avg_latency = max(avg_latency, d.average)
                  elif m.name.value == "OriginHealthPercentage":
                      for ts in m.timeseries:
                          for d in ts.data:
                              if d.average is not None:
                                  health_pct = min(health_pct, d.average)

              origins_audited += 1

              if avg_latency >= max_latency or health_pct < 100.0:
                  degraded_origins.append({
                      "frontdoor_profile": fd_profile,
                      "resource_group": rg_name,
                      "average_origin_latency_ms": round(avg_latency, 2),
                      "origin_health_percentage": round(health_pct, 2),
                      "threshold_latency_ms": max_latency,
                      "remediation": f"az afd origin list --profile-name {fd_profile} --resource-group {rg_name}"
                  })
          else:
              print("Azure credentials not provided; proceeding in dry-run mode.")
      except Exception as e:
          print(f"Error querying Azure Monitor metrics for Front Door {fd_profile}: {e}")

      Kestra.outputs({
          "origins_audited_count": origins_audited,
          "degraded_origins_count": len(degraded_origins),
          "has_degraded_origins": len(degraded_origins) > 0,
          "degraded_origins_details": json.dumps(degraded_origins)
      })

  - id: check_origin_latency
    type: io.kestra.plugin.core.flow.If
    description: Evaluates whether Azure Front Door origin response latency exceeded
      acceptable SLA thresholds.
    condition: "{{ outputs.scan_frontdoor_origin_telemetry.vars.has_degraded_origins
      and inputs.alert_on_latency }}"
    then:
      - id: notify_slack_origin_latency
        type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
        description: Dispatches high-priority incident notifications to Slack with Front
          Door profile names and origin response metrics.
        url: "{{ secret('SLACK_WEBHOOK_URL') }}"
        payload: |
          {
            "channel": "#azure-alerts",
            "username": "Azure-FrontDoor-Sentinel",
            "text": "CRITICAL: Azure Front Door origin latency degradation detected on profile `{{ inputs.frontdoor_profile_name }}`. Origins audited: `{{ outputs.scan_frontdoor_origin_telemetry.vars.origins_audited_count }}`, Degraded origins: `{{ outputs.scan_frontdoor_origin_telemetry.vars.degraded_origins_count }}`."
          }

  - id: log_frontdoor_status
    type: io.kestra.plugin.core.log.Log
    description: Records an operational audit trail of Azure Front Door origin
      health evaluations.
    message: "Azure Front Door audit completed for profile '{{
      inputs.frontdoor_profile_name }}'. Origins: {{
      outputs.scan_frontdoor_origin_telemetry.vars.origins_audited_count }},
      Degraded: {{
      outputs.scan_frontdoor_origin_telemetry.vars.degraded_origins_count }}."

  - id: export_frontdoor_manifest
    type: io.kestra.plugin.core.debug.Return
    description: Emits structured manifest of Azure Front Door origin performance
      and health percentages.
    format:
      profile_name: "{{ inputs.frontdoor_profile_name }}"
      resource_group: "{{ inputs.resource_group_name }}"
      origins_audited_count: "{{
        outputs.scan_frontdoor_origin_telemetry.vars.origins_audited_count }}"
      degraded_origins_count: "{{
        outputs.scan_frontdoor_origin_telemetry.vars.degraded_origins_count }}"
      details: "{{
        outputs.scan_frontdoor_origin_telemetry.vars.degraded_origins_details
        }}"

triggers:
  - id: scheduled_frontdoor_audit
    type: io.kestra.plugin.core.trigger.Schedule
    description: Automated schedule trigger auditing Azure Front Door origin health
      every 15 minutes.
    cron: "*/15 * * * *"

Global enterprise applications rely on Azure Front Door as a cloud-scale Content Delivery Network (CDN) and Layer 7 reverse proxy. Azure Front Door routes incoming client requests across global POPs to backend origins such as App Services, Kubernetes clusters, and regional application gateways. When backend origins suffer from CPU exhaustion, database deadlocks, or misconfigured health probe endpoints, origin response latency spikes and health percentages plunge. If undetected, edge caching fails to mask origin latency and end-user request timeouts increase. This sentinel interrogates Azure Monitor metrics across configured Front Door profiles on a scheduled cadence, measures origin latency percentiles and health probe scores, and alerts SRE teams on Slack before end-user service levels deteriorate.

How it works

Scheduled triggering: Executes every 15 minutes via automated cron trigger (*/15 * * * *) or triggers on-demand via manual execution.
Resource target binding: Binds to the designated Azure Front Door Standard/Premium profile resource identifier in the specified subscription.
Azure Monitor telemetry retrieval: Queries Azure Monitor metrics API for OriginLatency and OriginHealthPercentage over a 15-minute rolling window.
SLA threshold evaluation: Evaluates average origin response latency against max_origin_latency_ms and checks for health probe drops.
Conditional branching: An io.kestra.plugin.core.flow.If task evaluates whether origin latency or health breaches require intervention.
Slack incident dispatch: Emits rich incident cards to #azure-alerts detailing profile names, resource groups, observed latency in milliseconds, and CLI diagnostic commands.
Audit trail logging and return: Logs operational telemetry and returns a diagnostic JSON manifest for site reliability engineers.

Architecture diagram

flowchart TD
  A[scheduled_frontdoor_audit Schedule Trigger] --> B[scan_frontdoor_origin_telemetry Python Script]
  B --> C[check_origin_latency Flowable If]
  C -- Origin Degraded --> D[notify_slack_origin_latency Webhook]
  C -- Origin Healthy --> E[log_frontdoor_status Log]
  D --> E
  E --> F[export_frontdoor_manifest Return]

Use cases

Global origin degradation detection: Alert cloud infrastructure teams when backend microservices experience latency spikes behind Azure Front Door.
Health probe failure isolation: Detect origin health probe timeouts before Front Door removes origins from the active rotation pool.
Multi-region traffic shift verification: Verify that secondary disaster recovery origins remain warm and respond within acceptable latency thresholds.

Inputs

Name	Type	Default	Description
`azure_subscription_id`	`STRING`	`00000000-0000-0000-0000-000000000000`	Target Azure subscription ID hosting the Azure Front Door profile.
`resource_group_name`	`STRING`	`production-rg`	Name of the Azure resource group containing the Front Door profile.
`frontdoor_profile_name`	`STRING`	`prod-global-frontdoor`	Name of the Azure Front Door Standard/Premium profile to audit.
`max_origin_latency_ms`	`INT`	`1200`	Maximum allowable origin response latency in milliseconds before triggering performance alerts.
`alert_on_latency`	`BOOL`	`true`	Whether to dispatch automated Slack incident notifications when origin degradation is detected.

Expected outputs

{{ outputs.scan_frontdoor_origin_telemetry.vars.origins_audited_count }}: Number of Front Door profiles evaluated during the audit.
{{ outputs.scan_frontdoor_origin_telemetry.vars.degraded_origins_count }}: Number of profiles exhibiting origin latency or health degradations.
{{ outputs.scan_frontdoor_origin_telemetry.vars.degraded_origins_details }}: Serialized JSON array of degraded origins with latency metrics.
{{ outputs.export_frontdoor_manifest.value }}: Structured dictionary containing the consolidated Front Door performance report.

Prerequisites

Outbound HTTPS network access from Kestra execution workers to Azure Resource Manager and Azure Monitor endpoints.
Azure Service Principal credentials configured with Monitoring Reader role on the Front Door resource group.
Slack incoming webhook endpoint configured for Azure infrastructure alerting.

Secrets

AZURE_CLIENT_ID: Azure Service Principal Application (Client) ID.
AZURE_CLIENT_SECRET: Azure Service Principal client secret credential.
AZURE_TENANT_ID: Azure Directory (Tenant) ID.
SLACK_WEBHOOK_URL: Slack Incoming Webhook URL used to dispatch incident notifications.

Quick start

Define AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID, and SLACK_WEBHOOK_URL in your Kestra Secret storage.
Import this workflow into your Kestra instance under the company.cloud namespace.
Configure azure_subscription_id, resource_group_name, and frontdoor_profile_name to match your environment.
Run a manual execution to test Azure Monitor metric extraction and verify Slack incident broadcasting.

Common pitfalls and troubleshooting

Resource ID casing: Azure Resource IDs are case-sensitive in API queries; ensure provider names and resource types follow standard casing conventions.
Aggregation intervals: Azure Front Door metrics are aggregated at 1-minute and 5-minute intervals; ensure timespan windows span at least 15 minutes.
Role assignment scope: Ensure the service principal has Reader and Monitoring Reader roles on the specific Front Door resource or its resource group.

How to extend

Automated origin disablement: Add an automated step to temporarily disable degraded origins using az afd origin update --enabled-state Disabled.
PagerDuty on-call escalation: Escalate critical global CDN origin failures directly to on-call site reliability engineers via PagerDuty.
Synthetic origin probe test: Execute synthetic curl health checks directly against backend origin hostnames to bypass edge caching during incident investigations.

Azure Front Door Origin Response Latency Sentinel

How it works

Architecture diagram

Use cases

Inputs

Expected outputs

Prerequisites

Secrets

Quick start

Common pitfalls and troubleshooting

How to extend

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

New to Kestra?

Azure Front Door Origin Response Latency Sentinel

How it works

Architecture diagram

Use cases

Inputs

Expected outputs

Prerequisites

Secrets

Quick start

Common pitfalls and troubleshooting

How to extend

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

More Cloud Blueprints

OpenKruise AdvancedStatefulSet In-Place Update Stall Sentinel

HashiCorp Nomad Workload Allocation OOM Kill Sentinel

Object Storage Integrity and Streaming Replica Parity Audit

Alert on nginx 5xx Error Rate

Gate OIDC Signing-Key Rollovers with a Pinned JWKS

New Relic APM Apdex Score Degradation Sentinel

More Infrastructure Blueprints

Amazon SNS Subscription Delivery Failure and DLQ Sentinel

AWS VPC Flow Logs Rejected Traffic Anomaly Sentinel

AWS Step Functions State Machine Execution Failure Sentinel

Amazon SQS Message Age and Consumer Processing Lag Sentinel

AWS WAF Blocked Requests Surge and L7 Attack Sentinel

AWS SQS Dead Letter Queue Replay Gate

New to Kestra?