Azure AI Foundry Deployment Model Drift Gate

A production AI deployment's underlying model can change without the team that approved it ever finding out: a redeploy that quietly points at a newer model version, a shared deployment someone repointed at a different model entirely. Azure AI Foundry's own console shows the current configuration if someone opens it; it does not compare that configuration against what was actually approved or notify anyone when it drifts.

This blueprint was created by Mohit Jeswani.

How it works

1. get_deployment (io.kestra.plugin.azure.aifoundry.GetDeployment) fetches deployment_name's real configuration via the Azure AI Projects DeploymentsClient, authenticated with Entra ID (service principal). Read-only; nothing in Azure AI Foundry is changed. allowFailure: true covers a missing deployment or an auth error without failing the flow outright.
2. evaluate (io.kestra.plugin.core.output.OutputValues) compares the real modelName (and, when set, modelVersion) against expected_model_name/expected_model_version.
3. route (io.kestra.plugin.core.flow.If):
   • compliant: logs the real model, matching what was approved.
   • alert: posts the real model and the expected one to Slack.
4. summary logs the final severity, closing the governance cycle.
5. errors alerts Slack if the flow itself fails, for example an expired service principal secret.

What you get

• A direct read of the deployment's real model configuration, not a changelog or a deployment pipeline's own record of what it thinks is deployed.
• A missing deployment and a genuine model swap both surface, with distinct severities (CRITICAL vs. MODEL_DRIFT) so you know which one happened without reading logs.
• A read-only check with zero risk of creating, updating, or deleting a real deployment.

Who it's for

• AI platform and ML ops teams running production deployments on Azure AI Foundry who need to know the moment a deployment's model configuration drifts from what was approved, not discover it when a downstream application starts behaving differently.
• Teams evaluating the Azure plugin who want a worked example of aifoundry.GetDeployment used as a standing compliance check, the first blueprint in this repo to use the aifoundry sub-package of the Azure plugin at all.

Why orchestrate this with Kestra

Azure AI Foundry reports a deployment's configuration on request; it does not know what your organization approved, compare the two, or notify a Slack channel when they diverge. Kestra keeps expected_model_name/expected_model_version as flow inputs instead of institutional memory, turns a drift into a Slack page naming the exact model and version found, and keeps a full execution history of every check.

Prerequisites

• An Azure AI Foundry project with at least one model deployment.
• An Azure Entra ID service principal (tenantId, clientId, clientSecret) with read access to that project; API-key authentication is not supported by this specific client.
• A Slack incoming webhook URL.

This flow calls the Azure AI Foundry API directly over HTTPS; there is no official local Azure AI Foundry emulator, so get_deployment cannot be exercised against a local Docker container.

Secrets

• AZURE_TENANT_ID / AZURE_CLIENT_ID / AZURE_CLIENT_SECRET: the Entra ID service principal credentials used by get_deployment. In Kestra OSS, provide them as the base64-encoded environment variables SECRET_AZURE_TENANT_ID, SECRET_AZURE_CLIENT_ID, and SECRET_AZURE_CLIENT_SECRET.
• SLACK_WEBHOOK_URL: used by alert and the errors block. In Kestra OSS, provide it as the base64-encoded environment variable SECRET_SLACK_WEBHOOK_URL.

Inputs

Input	Default	Purpose
deployment_name	gpt-4o	Deployment to audit
expected_model_name	gpt-4o	Model name the deployment is approved to run
expected_model_version	`` (empty)	Model version to enforce; blank skips the version check

Outputs

• {{ outputs.get_deployment.name }} / .type / .configuration: the real deployment, as GetDeployment reports it (configuration.modelName, .modelVersion, .modelPublisher, .connectionName).
• {{ outputs.evaluate.values.actual_model_name }} / .actual_model_version: the real model identity found.
• {{ outputs.evaluate.values.severity }}: OK, MODEL_DRIFT, or CRITICAL.

Pitfalls

• GetDeployment only supports Entra ID (service principal) authentication; setting an apiKey on this task makes it throw immediately, by design, since the underlying Azure client has no API-key code path.
• configuration.modelName/.modelVersion are only populated when the deployment is a ModelDeployment; a different deployment type on the same endpoint reports unknown for both here rather than a false match.
• Leaving expected_model_version blank intentionally only enforces the model name; set a real version once your deployment's rollout process pins to specific versions.

Quick start

1. Add the AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and SLACK_WEBHOOK_URL secrets.
2. Set ai_foundry_endpoint and deployment_name to a real deployment.
3. Set expected_model_name/expected_model_version to match its current real configuration, run the flow, and confirm compliant logs the match.
4. Change expected_model_name to a different value to confirm alert fires as MODEL_DRIFT.
5. Restore the real expected values and enable the daily trigger.

How to extend

• Loop over several deployment_name values with io.kestra.plugin.core.flow.Loop to audit every production deployment in a project in one scheduled run.
• Add connectionName to the drift check alongside model name and version, to also catch a deployment silently repointed at a different backing connection.
• Chain an AI usage-cost task after a confirmed drift to estimate the cost impact of the unexpected model before paging the on-call owner.

Links

• Azure plugin: https://kestra.io/plugins/plugin-azure
• Azure AI Foundry deployments reference: https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/deployments-overview
• Slack incoming webhooks: https://api.slack.com/messaging/webhooks