Databricks Lakebase Trigger

Databricks Lakebase Trigger

Certified

Wait for query results on Databricks Lakebase and trigger a flow

Periodically mints a fresh Lakebase OAuth credential and polls a SQL query. The trigger fires when the query returns at least one row. Default interval is one minute. Use fetchType to control how matching rows are exposed on the execution (trigger.rows, trigger.row, or trigger.uri).

yaml
type: io.kestra.plugin.databricks.lakebase.Trigger

React to newly inserted rows

yaml
id: lakebase_new_rows_trigger
namespace: company.team

triggers:
  - id: on_new_order
    type: io.kestra.plugin.databricks.lakebase.Trigger
    workspaceHost: "{{ secret('DATABRICKS_HOST') }}"
    clientId: "{{ secret('DATABRICKS_CLIENT_ID') }}"
    clientSecret: "{{ secret('DATABRICKS_CLIENT_SECRET') }}"
    endpoint: "{{ secret('LAKEBASE_ENDPOINT_NAME') }}"
    database: "orders_db"
    sql: "SELECT * FROM orders WHERE status = 'pending'"
    interval: PT1M

tasks:
  - id: handle_new_order
    type: io.kestra.plugin.core.log.Log
    message: "New pending order: {{ trigger.rows }}"
Properties

OAuth M2M client ID

Service principal client ID (UUID). Also used as the Postgres username when connecting to Lakebase.

OAuth M2M client secret

Service principal OAuth secret used to mint a short-lived Lakebase database credential. Render from secrets.

Postgres database name

Database to open after authentication, e.g. databricks_postgres or orders_db

Lakebase endpoint resource name

Passed to generateDatabaseCredential. Format: projects//branches//endpoints/

SQL query to poll

The trigger fires when this query returns at least one row

Databricks workspace host

Workspace URL used by the Databricks SDK to mint a Lakebase database credential, e.g. https://.cloud.databricks.com

SQL to execute after a successful poll in the same transaction

Optional single statement used to mark processed rows so the next poll does not re-fire on the same data

Defaultfalse

Specifies whether a trigger is allowed to start a new execution even if a previous run is still in progress.

DefaultFETCH
Possible Values
STOREFETCHFETCH_ONENONE

Result fetching mode

Triggers default to FETCH. NONE is rejected because the trigger would never fire. Use FETCH, FETCH_ONE, or STORE.

Lakebase Postgres hostname

Optional. When omitted, the hostname is resolved from the endpoint via the Databricks Postgres API (status.hosts.host).

DefaultPT1M
Formatduration

Interval between polling.

The interval between 2 different polls of schedule, this can avoid to overload the remote system with too many calls. For most of the triggers that depend on external systems, a minimal interval must be at least PT30S. See ISO_8601 Durations for more information of available interval values.

Named parameter bindings

Map of parameter names to values. Use : name placeholders in SQL.

Default5432

Lakebase Postgres port

Defaults to 5432

Defaulttrue

Enable SSL

Lakebase requires SSL. Defaults to true.

SSL client certificate

PEM-encoded client certificate written to a temp file and passed to the Postgres driver as sslcert

SSL client key

PEM-encoded client key written to a temp file and passed to the Postgres driver as sslkey

SSL client key password

DefaultREQUIRE
Possible Values
DISABLEALLOWPREFERREQUIREVERIFY_CAVERIFY_FULL

SSL mode

Postgres sslmode. Defaults to REQUIRE, which is the Lakebase-recommended setting.

SSL root certificate

PEM-encoded CA certificate written to a temp file and passed to the Postgres driver as sslrootcert

SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

List of execution states after which a trigger should be stopped (a.k.a. disabled).

Time zone for temporal values

Defaulttrue

A condition that determines whether the trigger should run.

A Pebble expression evaluated at trigger time. The trigger fires only when the expression evaluates to a truthy value (true, a non-empty string, a non-zero number). Use this to gate trigger execution on dynamic runtime values such as execution labels, flow variables, or environment conditions.

First fetched row

Present when fetchType is FETCH_ONE

SubTypeobject

Fetched rows

Present when fetchType is FETCH

Number of fetched rows

Formaturi

Result file URI

Internal storage URI of the Ion file when fetchType is STORE

More Plugins in this Category

Metaplane
Metaplane

Run and read data-observability monitors from Metaplane.

Data

5 tasks

6 blueprints

CloudQuery
CloudQuery

Tasks that run CloudQuery commands to sync data through the CLI.

Data

2 tasks

4 blueprints