Unity Catalog - Tables List

Unity Catalog - Tables List

Certified

List Unity Catalog tables

Lists the tables of a schema that the caller can access. On schemas with many tables, set omitColumns: true and/or fetchType: STORE to keep the task output small.

yaml
type: io.kestra.plugin.databricks.unitycatalog.table.List

List the tables of a schema

yaml
id: databricks_uc_table_list
namespace: company.team

tasks:
  - id: list_tables
    type: io.kestra.plugin.databricks.unitycatalog.table.List
    host: "{{ secret('DATABRICKS_HOST') }}"
    authentication:
      token: "{{ secret('DATABRICKS_TOKEN') }}"
    catalogName: main
    schemaName: landing_zone

Store the table names of a large schema in internal storage, without the column definitions.

yaml
id: databricks_uc_table_list_store
namespace: company.team

tasks:
  - id: list_tables
    type: io.kestra.plugin.databricks.unitycatalog.table.List
    host: "{{ secret('DATABRICKS_HOST') }}"
    authentication:
      token: "{{ secret('DATABRICKS_TOKEN') }}"
    catalogName: main
    schemaName: landing_zone
    omitColumns: true
    fetchType: STORE
Properties

Catalog name

Name of the parent catalog.

Schema name

Name of the parent schema.

Databricks account identifier

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150

Databricks authentication configuration

This property allows to configure the authentication to Databricks, different properties should be set depending on the type of authentication and the cloud provider. All configuration options can also be set using the standard Databricks environment variables. Check the Databricks authentication guide for more information.

Definitions
authTypestring

Authentication type

azureClientIdstring

Azure client ID

azureClientSecretstring

Azure client secret

azureTenantIdstring

Azure tenant ID

clientIdstring

Client ID

clientSecretstring

Client secret

googleCredentialsstring

Google credentials JSON

googleServiceAccountstring

Google service account email

passwordstring

Password

tokenstring

Databricks personal access token

usernamestring

Username

Databricks configuration file, use this if you don't want to configure each Databricks account properties one by one

DefaultFETCH
Possible Values
STOREFETCHFETCH_ONENONE

Fetch type

How the tables are returned:

  • FETCH: all the tables are available in the tables output (default).
  • FETCH_ONE: only the first table is available in the table output.
  • STORE: the tables are written to Kestra internal storage and exposed through the uri output.
  • NONE: nothing is returned except size.

Databricks host

Omit columns

Do not return the column definitions of each table. Recommended when only the table names are needed. Defaults to false.

Number of tables

First table as returned by Databricks

Only populated if using fetchType=FETCH_ONE.

SubTypeobject

Tables as returned by Databricks

Only populated if using fetchType=FETCH.

Formaturi

Kestra internal storage URI of the stored tables

Only populated if using fetchType=STORE. The file is in Ion format, one table per row.

More Plugins in this Category

Metaplane
Metaplane

Run and read data-observability monitors from Metaplane.

Data

5 tasks

6 blueprints

CloudQuery
CloudQuery

Tasks that run CloudQuery commands to sync data through the CLI.

Data

2 tasks

4 blueprints