Azure SubmitCommandJob
CertifiedSubmit a command job to an Azure Machine Learning compute target
Azure SubmitCommandJob
Submit a command job to an Azure Machine Learning compute target
Submits a single command job (e.g. a training script) to a compute cluster or instance and, by default, waits for it to reach a terminal state, exposing MLflow-backed metrics as task outputs. Killing the Kestra execution cancels the underlying Azure ML job. Defaults: wait=true, checkFrequency.interval=PT10S, checkFrequency.maxDuration=PT1H, cancelOnTimeout=true.
type: io.kestra.plugin.azure.ml.SubmitCommandJobExamples
id: azure_ml_submit_command_job
namespace: company.team
tasks:
- id: train
type: io.kestra.plugin.azure.ml.SubmitCommandJob
tenantId: "{{ secret('AZURE_TENANT_ID') }}"
clientId: "{{ secret('AZURE_CLIENT_ID') }}"
clientSecret: "{{ secret('AZURE_CLIENT_SECRET') }}"
subscriptionId: "{{ secret('AZURE_SUBSCRIPTION_ID') }}"
resourceGroupName: ml-rg
workspaceName: ml-workspace
computeName: cpu-cluster
environmentId: "azureml:AzureML-sklearn-1.5:1"
command: "python train.py --epochs 10"
- id: check_accuracy
type: io.kestra.plugin.core.execution.Fail
condition: "{{ outputs.train.metrics['accuracy'] < 0.9 }}"
Properties
command *string
Command
Shell command executed on the compute target, e.g. python train.py --epochs 10
computeName *string
Compute target name
Name of an existing Azure Machine Learning compute cluster or compute instance
resourceGroupName *string
Resource group name
Resource group containing the Machine Learning workspace
subscriptionId *string
Subscription ID
Azure subscription GUID that owns the Machine Learning workspace
tenantId *string
Azure AD tenant ID (GUID)
workspaceName *string
Workspace name
Name of the Azure Machine Learning workspace
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150cancelOnTimeout booleanstring
trueCancel the job on timeout
When wait=true and checkFrequency.maxDuration is exceeded, cancel the Azure ML job before failing the task; defaults to true
checkFrequency
{
"maxDuration": "PT1H",
"interval": "PT10S"
}Polling frequency
Interval and max duration used when wait=true
io.kestra.plugin.azure.ml.SubmitCommandJob-CheckFrequency
PT10SPolling interval
Delay between status checks; defaults to PT10S. Azure ML jobs run for minutes to hours, so a tighter interval only adds needless ARM API calls
PT1HMax wait duration
Stop polling and fail after this duration; defaults to PT1H
clientId string
Client ID of the Azure AD application
Application (client) ID used for service principal authentication.
clientSecret string
Client secret for the Azure AD application
Secret value associated with the client ID; store in a Kestra secret.
displayName string
Display name
Human-readable run name shown in Azure ML Studio
environmentId string
Environment ID
Registered environment reference, e.g. azureml: AzureML-sklearn-1.5: 1; required by Azure Machine Learning to run the command
experimentName string
Experiment name
Groups this run under an experiment in Azure ML Studio
inputs object
Data inputs
Named folder inputs available to the command as ${{inputs.<name>}}; keys are input names, values are storage URIs such as azureml://datastores/<store>/paths/<path>.
instanceCount integerstring
1Instance count
Number of compute nodes to use for this job; defaults to 1
name string
Job name
Unique job name within the workspace; a random UUID is generated when not set
outputs object
Data outputs
Named folder outputs the command writes to via ${{outputs.<name>}}; keys are output names, values are destination storage URIs.
pemCertificate string
PEM-encoded certificate content for client authentication
PEM text for certificate-based auth; alternative to clientSecret.
wait booleanstring
trueWait for completion
If true (default), poll the job status until it reaches a terminal state and collect metrics
Outputs
jobName string
Job name
Name of the submitted Azure Machine Learning job
metrics object
Metrics
Metrics logged by the run, keyed by metric name. Always empty when wait=false, since the job has not necessarily finished logging anything yet.
Azure Machine Learning logs job metrics through MLflow, not through the ARM control-plane API used for everything else in this task. This value is fetched best-effort by reading the workspace's MLflow tracking URI and calling its REST API directly with the same Azure AD bearer token used to authenticate this task. If that call fails (e.g. the service principal lacks the required scope, or the endpoint is unreachable), a warning is logged and this field is an empty map — the task does not fail because of it.
outputs object
Outputs
Named job outputs, keyed by output name, pointing to their storage URI
status string
NOT_STARTEDSTARTINGPROVISIONINGPREPARINGQUEUEDRUNNINGFINALIZINGCANCEL_REQUESTEDCOMPLETEDFAILEDCANCELEDNOT_RESPONDINGPAUSEDUNKNOWNJob status
Terminal or last-observed job status
studioUrl string
Studio URL
Deep link to the run in Azure ML Studio
