Create a Pipeline run from an Azure Data Factory.
Launch an Azure DataFactory pipeline from Kestra. Data Factory contains a series of interconnected systems that provide a complete end-to-end platform for data engineers.
type: "io.kestra.plugin.azure.datafactory.createrun"Examples
id: azure_datafactory_create_run
namespace: company.team
tasks:
- id: create_run
type: io.kestra.plugin.azure.datafactory.CreateRun
factoryName: exampleFactoryName
pipelineName: examplePipeline
resourceGroupName: exampleResourceGroup
subscriptionId: 12345678-1234-1234-12345678abc
tenantId: "{{ secret('DATAFACTORY_TENANT_ID') }}"
clientId: "{{ secret('DATAFACTORY_CLIENT_ID') }}"
clientSecret: "{{ secret('DATAFACTORY_CLIENT_SECRET') }}"
Properties
subscriptionId*Requiredstring
Subscription ID
tenantId*Requiredstring
Tenant ID
checkFrequencyNon-dynamic
{
"maxDuration": "PT1H",
"interval": "PT5S"
}Check the frequency configuration.
io.kestra.plugin.azure.datafactory.CreateRun-CheckFrequency
PT5SdurationFrequency at which Kestra checks if the pipeline has finished.
PT1HdurationMaximum duration of the task until timing out the task.
clientIdstring
Client ID
Client ID of the Azure service principal. If you don't have a service principal, refer to create a service principal with Azure CLI.
clientSecretstring
Client Secret
Service principal client secret. The tenantId, clientId and clientSecret of the service principal are required for this credential to acquire an access token.
factoryNamestring
Factory name
parametersobject
{}Pipeline parameters.
pemCertificatestring
PEM Certificate
Your stored PEM certificate.
The tenantId, clientId and clientCertificate of the service principal are required for this credential to acquire an access token.
pipelineNamestring
Pipeline name
resourceGroupNamestring
Resource Group name
waitbooleanstring
trueWait for the end of the run.
Allowing to capture job status & logs.
Outputs
runIdstring
Run ID
The ID of the pipeline run created in Azure Data Factory
uristring
uriURI of a kestra internal storage file containing the activities and their inputs/outputs.
Metrics
activities.countcounter
The total number of activities in the pipeline run.
pipeline.duration.mstimer
The duration of the pipeline run in milliseconds.
