SubmitRun

yaml
type: "io.kestra.plugin.databricks.job.SubmitRun"

Examples

yaml
id: databricks_job_submit_run
namespace: company.team

tasks:
  - id: submit_run
    type: io.kestra.plugin.databricks.job.SubmitRun
    host: "{{ secret('DATABRICKS_HOST') }}"
    authentication:
      token: "{{ secret('DATABRICKS_TOKEN') }}"
    runTasks:
      - existingClusterId: <your-cluster>
        taskKey: pysparkTask
        sparkPythonTask:
          pythonFile: /Shared/hello.py
          sparkPythonTaskSource: WORKSPACE
    waitForCompletion: PT5M

Properties

runTasks *array

Min items1

Definitions

io.kestra.plugin.databricks.job.SubmitRun-RunSubmitTaskSetting

dependsOnarray

SubTypestring

existingClusterIdstring

librariesarray

io.kestra.plugin.databricks.job.task.LibrarySetting

cran

io.kestra.plugin.databricks.job.task.LibrarySetting-CranSetting

_packagestring

repostring

eggstring

jarstring

maven

io.kestra.plugin.databricks.job.task.LibrarySetting-MavenSetting

coordinatesstring

exclusionsarray

SubTypestring

repostring

pypi

io.kestra.plugin.databricks.job.task.LibrarySetting-PypiSetting

_packagestring

repostring

whlstring

notebookTask

io.kestra.plugin.databricks.job.task.NotebookTaskSetting

baseParametersstringobject

SubTypestring

notebookPathstring

sourcestring

Possible Values

GITWORKSPACE

pipelineTask

io.kestra.plugin.databricks.job.task.PipelineTaskSetting

fullRefreshbooleanstring

pipelineIdstring

pythonWheelTask

io.kestra.plugin.databricks.job.task.PythonWheelTaskSetting

entryPointstring

namedParametersstringobject

SubTypestring

packageNamestring

parametersstringarray

runJobTask

io.kestra.plugin.databricks.job.task.RunJobTaskSetting

jobIdstring

jobParametersobject

sparkJarTask

io.kestra.plugin.databricks.job.task.SparkJarTaskSetting

jarUristring

mainClassNamestring

parametersstringarray

sparkPythonTask

io.kestra.plugin.databricks.job.task.SparkPythonTaskSetting

pythonFile*string

sparkPythonTaskSource*string

Possible Values

GITWORKSPACE

parametersstringarray

sparkSubmitTask

io.kestra.plugin.databricks.job.task.SparkSubmitTaskSetting

parametersstringarray

taskKeystring

timeoutSecondsinteger

accountId string

authentication

Definitions

io.kestra.plugin.databricks.AbstractTask-AuthenticationConfig

authTypestring

azureClientIdstring

azureClientSecretstring

azureTenantIdstring

clientIdstring

clientSecretstring

googleCredentialsstring

googleServiceAccountstring

passwordstring

tokenstring

usernamestring

configFile string

host string

runName string

waitForCompletion string

Formatduration

Outputs

runId integer

runURI string

Formaturi

Core Plugins and tasks

Tasks that provide Kestra's built-in orchestration, I/O, and observability capabilities.

CoreStorageScript

Stripe

Tasks that interact with Stripe customers, payments, balances, and webhooks.

Tool

AWS

AWS plugin for Kestra

AlertingCloudMessagingStorageDatabaseScriptToolTransformation

SubmitRun

More Plugins in this Category

Core Plugins and tasks

Stripe

AWS

1.1.1