CreateCluster
Create a Databricks cluster.
type: "io.kestra.plugin.databricks.cluster.CreateCluster"Create a Databricks cluster with one worker.
id: databricks_create_cluster
namespace: company.team
tasks:
- id: create_cluster
type: io.kestra.plugin.databricks.cluster.CreateCluster
authentication:
token: <your-token>
host: <your-host>
clusterName: kestra-demo
nodeTypeId: n2-highmem-4
numWorkers: 1
sparkVersion: 13.0.x-scala2.12
The name of the cluster.
The Spark version.
Databricks account identifier.
Databricks authentication configuration.
This property allows to configure the authentication to Databricks, different properties should be set depending on the type of authentication and the cloud provider. All configuration options can also be set using the standard Databricks environment variables. Check the Databricks authentication guide for more information.
Databricks configuration file, use this if you don't want to configure each Databricks account properties one by one.
Databricks host.
The type of node, the value depends on the cloud provider.
The cluster identifier.
The cluster state.
The cluster URI on the Databricks console.