Google Cloud BigQuery TableMetadata
Certified
Get BigQuery table metadata
Google Cloud BigQuery TableMetadata
Get BigQuery table metadata
Retrieves metadata for a table or partition. Can return empty output instead of failing when the table is missing.
type: io.kestra.plugin.gcp.bigquery.TableMetadataExamples
id: gcp_bq_table_metadata
namespace: company.team
tasks:
- id: table_metadata
type: io.kestra.plugin.gcp.bigquery.TableMetadata
projectId: my-project
dataset: my-dataset
table: my-table
Installation
Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:
kestractl plugins get io.kestra.plugin:plugin-gcp --compatible-for $KESTRA_VERSION For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.
Properties
dataset *string
Dataset ID
Target dataset name
table *string
Table ID
Target table name
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150ifNotExists string
ERRORERRORSKIPMissing table policy
ERROR by default. Use SKIP to return an empty output when the table does not exist.
impersonatedServiceAccount string
The GCP service account to impersonate
location string
Dataset location
Optional BigQuery location for created or targeted resources. Experimental and may change; see BigQuery dataset location documentation.
projectId string
The GCP project ID
retryAuto
Automatic BigQuery retry policy
Optional custom retry policy for retryable BigQuery errors. If unset, uses an exponential backoff starting at 5s (per-attempt interval capped at 60m) with a maximum of 10 attempts and no overall time limit, so long-running queries are still retried. Set maxDuration or task timeout to cap the total time.
Constant retry
Retry with a fixed delay between attempts.
durationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION>= 1durationfalseExponential retry
Retry with exponentially increasing delays between attempts.
durationdurationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION>= 1durationfalseRandom retry
Retry with a random delay within a configurable range between attempts.
durationdurationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION>= 1durationfalseretryMessages array
["due to concurrent update","Retrying the job may solve the problem","Retrying may solve the problem","[CONNECTION_ERROR]","This stub requires a minimum remaining timeout"]Retry message substrings
Case-insensitive substrings that, if found in the error message, trigger an automatic retry.
retryReasons array
["rateLimitExceeded","jobBackendError","backendError","internalError","jobInternalError"]Retry reasons
BigQuery error reasons (matched case-insensitively) that trigger an automatic retry.
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serviceAccount string
The GCP service account
When unset or rendered to an empty value, Application Default Credentials are used (e.g. GOOGLE_APPLICATION_CREDENTIALS or the runtime's attached identity).
workloadIdentityAudience string
Audience of the Workload Identity Federation provider
The full resource name of the workload identity pool provider, //iam.googleapis.com/projects/<number>/locations/global/workloadIdentityPools/<pool>/providers/<provider>.
Required with workloadIdentityToken, and must match the aud claim of the token.
workloadIdentityToken string
OIDC token for Workload Identity Federation
When set together with workloadIdentityAudience, the token is exchanged at Google STS for federated credentials and no service account key is needed: Google verifies it against the identity provider that issued it.
Use {{ oidcToken(audience='…') }} to have Kestra Enterprise mint one for the running execution.
Combine with impersonatedServiceAccount to act as a service account the pool may impersonate, and set projectId explicitly since no key file carries it.
Cannot be combined with serviceAccount.
Outputs
creationTime string
date-timeCreation time
datasetId string
Dataset ID
definition
Table definition
io.kestra.plugin.gcp.bigquery.models.TableDefinition-Output
io.kestra.plugin.gcp.bigquery.models.ExternalTableDefinition-Output
CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORCio.kestra.plugin.gcp.bigquery.models.MaterializedViewDefinition-Output
date-timedurationio.kestra.plugin.gcp.bigquery.models.Schema-Output
io.kestra.plugin.gcp.bigquery.models.Field-Output
NULLABLEREQUIREDREPEATEDBOOLINT64FLOAT64NUMERICBIGNUMERICSTRINGBYTESSTRUCTARRAYTIMESTAMPDATETIMEDATETIMEGEOGRAPHYJSONINTERVALRANGEio.kestra.plugin.gcp.bigquery.models.StandardTableDefinition-Output
io.kestra.plugin.gcp.bigquery.models.RangePartitioning-Output
com.google.cloud.bigquery.StandardTableDefinition-StreamingBuffer
io.kestra.plugin.gcp.bigquery.models.TimePartitioning-Output
durationDAYHOURMONTHYEARTABLEVIEWMATERIALIZED_VIEWEXTERNALMODELio.kestra.plugin.gcp.bigquery.models.ViewDefinition-Output
io.kestra.plugin.gcp.bigquery.models.UserDefinedFunction-Output
INLINEFROM_URIdescription string
Description
encryptionConfiguration
Encryption configuration
io.kestra.plugin.gcp.bigquery.models.EncryptionConfiguration-Output
etag string
Table etag
expirationTime string
date-timeExpiration time
If absent, the table persists indefinitely; expired tables are deleted
friendlyName string
Friendly name
generatedId string
Service-generated table id
labels object
Labels
lastModifiedTime string
date-timeLast modified time
numBytes integer
Size in bytes
numLongTermBytes integer
Long-term storage bytes
Bytes billed at long-term storage rates
numRows integer
Row count
projectId string
Project ID
requirePartitionFilter boolean
Require partition filter
True when queries must specify a partition filter for partition elimination
selfLink string
Self link URL
API URL for get or update requests
table string
Table name