How to use the Azure EE plugin

Run tasks on Azure Batch or natively on Azure Virtual Machines, manage Azure Blob Storage, and ship execution logs to Azure Monitor and Blob Storage from Kestra flows.

Authentication

Azure Batch tasks and the task runner use account (required) and endpoint (required, the Batch endpoint URL). Set accessKey for shared key authentication.

Azure Blob Storage tasks and the storage log exporter use endpoint (required, the Blob service endpoint URL). Authenticate with one of: connectionString, sharedKeyAccountName + sharedKeyAccountAccessKey, sasToken, or service principal (tenantId + clientId + clientSecret or pemCertificate).

Azure Monitor log exporter uses endpoint (required, the Data Collection Endpoint URL). Authenticate with a service principal (tenantId + clientId + clientSecret or pemCertificate), or rely on the Azure default credential chain if none are set.

Azure Virtual Machine task runner uses subscriptionId (required) and, for service-principal authentication, tenantId + clientId + clientSecret (all secrets). When the three are left unset, it falls back to DefaultAzureCredential (e.g. a managed identity attached to the Kestra Worker).

Store secrets in secrets and apply connection properties globally with plugin defaults.

Tasks

batch.job.Create submits and monitors an Azure Batch job — set account, endpoint, poolId, job, and tasks (all required). Optionally set maxDuration (no timeout if unset), completionCheckInterval (default PT1S), delete (default true), and resume (default true). Outputs vars and outputFiles.

storage.blob.List lists blobs in a container — set endpoint (required). Optionally set container, prefix, regexp, delimiter, and filter (FILES, DIRECTORY, or BOTH, default FILES). Output includes blobs.

storage.blob.SharedAccess generates a SAS URI for a blob — set endpoint, expirationDate (ISO 8601), and permissions (list of READ, ADD, CREATE, WRITE, DELETE, etc., all required). Output includes uri.

Task runners

runner.Batch runs task scripts inside Azure Batch jobs — set endpoint and poolId (both required). For file staging (inputFiles, outputFiles, namespaceFiles), set blobStorage.containerName (required within blobStorage) along with blob authentication properties. Optionally configure blobStorage.endpoint, blobStorage.connectionString, or blobStorage.sharedKeyAccountName/sharedKeyAccountAccessKey. Set registry for private container image pulls.

Key behavioral properties: waitUntilCompletion (default PT1H), completionCheckInterval (default PT5S), delete (default true), resume (default true), streamLogs (default false).

runner.VirtualMachine runs task scripts natively (no container, containerImage is ignored) on an Azure VM via the Run Command API. Set subscriptionId and resourceGroupName (both required). To provision a new VM, also set region, vmSize, and subnetId (the full resource ID of an already existing subnet the VM's network interface joins). The runner never creates a virtual network of its own: the subnet given in subnetId must already exist. To reuse an already existing VM instead, set vmName. A vmName-targeted VM must already exist, the runner throws an actionable error if it doesn't, and it never creates, stops, or deletes that VM since it isn't the one that provisioned it: stopVm and deleteVm are ignored for it. The VM gets no public IP and no inbound network rules, since Run Command never needs one. For file staging (inputFiles, outputFiles, namespaceFiles, {{ outputDir }}), set blobStorage the same way as for runner.Batch: the runner stages files through Blob Storage directly and hands the VM a short-lived, container-scoped SAS token so its script can pull/push with plain curl, since a plain VM image can't be assumed to have the Azure CLI installed.

Because the VM has no NAT/egress configured on its subnet, using blobStorage requires the subnet given in subnetId to provide some other outbound path to *.blob.core.windows.net, for example a NAT gateway on the subnet, or a Blob Storage private endpoint or service endpoint on the resource group. Azure is retiring default outbound internet access for virtual networks, so without one of these in place, file staging can fail silently.

Key behavioral properties: stopVm (default true, deallocates the VM to stop compute billing), deleteVm (default true, also removes the VM's network interface and OS disk), waitUntilCompletion (default PT1H), completionCheckInterval (default PT5S), resume (default true, only applies when vmName is unset), instanceReadyTimeout (default PT5M). Because Azure's Run Command call is synchronous (no command ID to reattach to, unlike AWS SSM), a Worker restart mid-run always re-issues the command against the resumed VM rather than reattaching, so commands should tolerate being run more than once.

Log exporters

monitor.LogExporter ships execution logs to Azure Monitor via the Log Ingestion API — set endpoint, ruleId (Data Collection Rule ID), and streamName (all required). Optionally set tenantId, clientId, clientSecret or pemCertificate for service principal auth, and chunk (logs per request, default 1000).

storage.LogExporter ships execution logs to Azure Blob Storage — set endpoint and containerName (both required). Optionally set format (default JSON, also accepts ION), maxLinesPerFile (default 100000), logFilePrefix (default kestra-log-file), and chunk (default 1000).