query
Execute a SQL query on a Databricks cluster.
Execute a SQL query on a Databricks cluster.
Execute a SQL query on a Databricks cluster.
See Retrieve the connection details in the Databricks documentation to discover how to retrieve the needed configuration properties. We're using the Databricks JDBC driver to execute a Query, see https://docs.databricks.com/integrations/jdbc-odbc-bi.html#jdbc-driver-capabilities for its capabilities.
Due to current limitation of the JDBC driver with Java 21, Arrow is disabled, performance may be impacted, see here and here from Databricks status on Java 21 support.
type: "io.kestra.plugin.databricks.sql.query"Examples
id: databricks_sql_query
namespace: company.team
tasks:
- id: sql_query
type: io.kestra.plugin.databricks.sql.Query
accessToken: "{{ secret('DATABRICKS_TOKEN') }}"
host: <your-host>
httpPath: <your-httpPath>
sql: SELECT 1
Properties
host*Requiredstring
Databricks host.
httpPath*Requiredstring
Databricks cluster HTTP Path.
To retrieve the HTTP Path, go to your Databricks cluster, click on Advanced options then, click on JDBC/ODBC. See Retrieve the connection details for more details.
sql*Requiredstring
SQL query to be executed.
accessTokenstring
Databricks access token.
catalogstring
propertiesobject
schemastring
timeZoneIdstring
The time zone id to use for date/time manipulation. Default value is the worker default zone id.
Outputs
sizeinteger
The number of fetched rows.
uristring
uriThe URI of the result file in Kestra's internal storage (.ion file i.e. Amazon Ion text format).
Metrics
fetch.sizecounter
Query result size