SubmitRun

yaml
type: "io.kestra.plugin.databricks.job.SubmitRun"

Examples

yaml
id: databricks_job_submit_run
namespace: company.team

tasks:
  - id: submit_run
    type: io.kestra.plugin.databricks.job.SubmitRun
    host: "{{ secret('DATABRICKS_HOST') }}"
    authentication:
      token: "{{ secret('DATABRICKS_TOKEN') }}"
    runTasks:
      - existingClusterId: <your-cluster>
        taskKey: pysparkTask
        sparkPythonTask:
          pythonFile: /Shared/hello.py
          sparkPythonTaskSource: WORKSPACE
    waitForCompletion: PT5M

Properties

runTasks*array

Min items1

Definitions

io.kestra.plugin.databricks.job.SubmitRun-RunSubmitTaskSetting

dependsOnarray

SubTypestring

existingClusterIdstring

librariesarray

io.kestra.plugin.databricks.job.task.LibrarySetting

cran

io.kestra.plugin.databricks.job.task.LibrarySetting-CranSetting

_packagestring

repostring

eggstring

jarstring

maven

io.kestra.plugin.databricks.job.task.LibrarySetting-MavenSetting

coordinatesstring

exclusionsarray

SubTypestring

repostring

pypi

io.kestra.plugin.databricks.job.task.LibrarySetting-PypiSetting

_packagestring

repostring

whlstring

notebookTask

io.kestra.plugin.databricks.job.task.NotebookTaskSetting

baseParametersstringobject

SubTypestring

notebookPathstring

sourcestring

Possible Values

GITWORKSPACE

pipelineTask

io.kestra.plugin.databricks.job.task.PipelineTaskSetting

fullRefreshbooleanstring

pipelineIdstring

pythonWheelTask

io.kestra.plugin.databricks.job.task.PythonWheelTaskSetting

entryPointstring

namedParametersstringobject

SubTypestring

packageNamestring

parametersstringarray

runJobTask

io.kestra.plugin.databricks.job.task.RunJobTaskSetting

jobIdstring

jobParametersobject

sparkJarTask

io.kestra.plugin.databricks.job.task.SparkJarTaskSetting

jarUristring

mainClassNamestring

parametersstringarray

sparkPythonTask

io.kestra.plugin.databricks.job.task.SparkPythonTaskSetting

pythonFile*string

sparkPythonTaskSource*string

Possible Values

GITWORKSPACE

parametersstringarray

sparkSubmitTask

io.kestra.plugin.databricks.job.task.SparkSubmitTaskSetting

parametersstringarray

taskKeystring

timeoutSecondsinteger

accountIdstring

authentication

Definitions

io.kestra.plugin.databricks.AbstractTask-AuthenticationConfig

authTypestring

azureClientIdstring

azureClientSecretstring

azureTenantIdstring

clientIdstring

clientSecretstring

googleCredentialsstring

googleServiceAccountstring

passwordstring

tokenstring

usernamestring

configFilestring

hoststring

runNamestring

waitForCompletionstring

Formatduration

Outputs

runIdinteger

runURIstring

Formaturi

Core Plugins and tasks

Tasks that provide Kestra's built-in orchestration, I/O, and observability capabilities.

CoreStorageScript

PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Database

SurrealDB

Tasks that run SurrealQL queries against SurrealDB.

Database

SubmitRun

More Plugins in this Category

Core Plugins and tasks

PostgreSQL

SurrealDB

1.1.1