Microsoft Fabric RunSparkJob
CertifiedRun a Microsoft Fabric Spark Job Definition
Microsoft Fabric RunSparkJob
Run a Microsoft Fabric Spark Job Definition
Submits a Fabric Spark Job Definition using the item job scheduler API and optionally waits for completion. Defaults: wait=true, pollFrequency=PT5S, timeout=PT1H.
type: io.kestra.plugin.microsoft.fabric.data.engineering.RunSparkJobExamples
id: fabric_run_spark_job
namespace: company.team
tasks:
- id: run_spark_job
type: io.kestra.plugin.microsoft.fabric.data.engineering.RunSparkJob
tenantId: "{{ secret('FABRIC_TENANT_ID') }}"
clientId: "{{ secret('FABRIC_CLIENT_ID') }}"
clientSecret: "{{ secret('FABRIC_CLIENT_SECRET') }}"
workspaceId: "your-workspace-id"
sparkJobDefinitionId: "your-spark-job-definition-id"
Properties
sparkJobDefinitionId *string
Spark Job Definition ID
Microsoft Fabric Spark Job Definition item GUID within the workspace
workspaceId *string
Workspace ID
Microsoft Fabric workspace GUID
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150clientId string
Client ID
Service principal client (application) ID
clientSecret string
Client Secret
Service principal client secret. When both clientId and clientSecret are provided, service principal authentication is used; otherwise DefaultAzureCredential is used.
parameters object
{}Execution parameters
Key/value parameters passed to the job execution
pollFrequency string
PT5SPoll frequency
Interval between status checks when wait=true
tenantId string
Tenant ID
Azure Active Directory tenant ID for authentication
timeout string
PT1HTimeout
Maximum time to wait for completion; throws TimeoutException when exceeded
wait booleanstring
trueWait for completion
Poll until the job reaches a terminal state; returns the job instance ID immediately when false
Outputs
jobInstanceId string
Job instance ID
Unique ID of the Spark job instance
status string
Final status
Terminal status of the Spark job
