Google Cloud UpdateTable

Google Cloud UpdateTable
Certified
Update BigQuery table metadata

Updates an existing table’s labels, description, friendly name, encryption, expiration, and partition filter requirement while preserving the current definition unless overridden.

yaml
type: io.kestra.plugin.gcp.bigquery.UpdateTable

Examples

yaml
id: gcp_bq_update_table
namespace: company.team

tasks:
  - id: update_table
    type: io.kestra.plugin.gcp.bigquery.UpdateTable
    projectId: my-project
    dataset: my-dataset
    table: my-table
    expirationDuration: PT48H

Properties

dataset *string

Dataset ID

Target dataset name

table *string

Table ID

Target table name

description string

Table description

encryptionConfiguration

Encryption configuration

Optional CMEK for the table

Definitions

io.kestra.plugin.gcp.bigquery.models.EncryptionConfiguration

kmsKeyNamestring

The KMS key name.

expirationDuration string

Expiration duration

Duration from now until table expiration; if unset, table persists

friendlyName string

Table display name

impersonatedServiceAccount string

The GCP service account to impersonate

labels object

SubTypestring

Table labels

location string

Dataset location

Optional BigQuery location for created or targeted resources. Experimental and may change; see BigQuery dataset location documentation.

pluginDefaultsRef string

Reference (ref) of the pluginDefaults to apply to this task.

projectId string

The GCP project ID

requirePartitionFilter booleanstring

Require partition filter

If true, queries must include a partition filter for partitioned tables

retryAuto

Automatic BigQuery retry policy

Optional custom retry policy for retryable BigQuery errors. If unset, uses an exponential backoff starting at 5s (per-attempt interval capped at 60m), with a total duration of up to 15m and a maximum of 10 attempts.

Definitions

Constant retry

Retry with a fixed delay between attempts.

interval*string

Formatduration

type*object

behaviorstring

DefaultRETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

maxAttemptsinteger

Minimum>= 1

maxDurationstring

Formatduration

warningOnRetryboolean

Defaultfalse

Exponential retry

Retry with exponentially increasing delays between attempts.

interval*string

Formatduration

maxInterval*string

Formatduration

type*object

behaviorstring

DefaultRETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

delayFactornumber

maxAttemptsinteger

Minimum>= 1

maxDurationstring

Formatduration

warningOnRetryboolean

Defaultfalse

Random retry

Retry with a random delay within a configurable range between attempts.

maxInterval*string

Formatduration

minInterval*string

Formatduration

type*object

behaviorstring

DefaultRETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

maxAttemptsinteger

Minimum>= 1

maxDurationstring

Formatduration

warningOnRetryboolean

Defaultfalse

retryMessages array

SubTypestring

Default["due to concurrent update","Retrying the job may solve the problem","Retrying may solve the problem"]

Retry message substrings

Case-insensitive substrings that, if found in the error message, trigger an automatic retry

retryReasons array

SubTypestring

Default["rateLimitExceeded","jobBackendError","backendError","internalError","jobInternalError"]

Retry reasons

BigQuery error reasons that trigger an automatic retry; evaluated against error reason strings

scopes array

SubTypestring

Default["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used

serviceAccount string

The GCP service account

tableDefinition

Table definition

The table definition (standard, external, view, etc.). Required when creating a table; optional when updating, where the current definition is preserved unless overridden.

Definitions

io.kestra.plugin.gcp.bigquery.models.TableDefinition

externalTableDefinition

The external table definition if the type is EXTERNAL

io.kestra.plugin.gcp.bigquery.models.ExternalTableDefinition

autodetectbooleanstring

Whether automatic detection of schema and format options should be performed

compressionstring

The compression type of the data source

formatTypestring

Possible Values

CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORC

The source format, and possibly some parsing options, of the external data

ignoreUnknownValuesbooleanstring

Whether BigQuery should allow extra values that are not represented in the table schema

If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result.

maxBadRecordsintegerstring

The maximum number of bad records that BigQuery can ignore when reading data

If the number of bad records exceeds this value, an invalid error is returned in the job result.

sourceUrisarray

SubTypestring

The fully-qualified URIs that point to your data in Google Cloud Storage

Each URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

materializedViewDefinition

The materialized view definition if the type is MATERIALIZED_VIEW

io.kestra.plugin.gcp.bigquery.models.MaterializedViewDefinition

enableRefreshbooleanstring

Whether automatic refresh is enabled for the materialized view when the base table is updated

lastRefreshDatestring

Formatdate-time

Date when this materialized view was last refreshed

querystring

The query whose result is persisted

refreshIntervalstring

The maximum frequency at which this materialized view will be refreshed

schema

The table's schema

io.kestra.plugin.gcp.bigquery.models.Schema

fieldsarray

The fields associated with this schema.

io.kestra.plugin.gcp.bigquery.models.Field

descriptionstring

The field description

modestring

Possible Values

NULLABLEREQUIREDREPEATED

The field mode

By default, Field.Mode.NULLABLE is used.

namestring

The field name

policyTags

The policy tags for the field

subFieldsarray

The list of sub-fields if type is a record/struct type (StandardSQLTypeName.STRUCT). Null otherwise.

typestring

Possible Values

BOOLINT64FLOAT64NUMERICBIGNUMERICSTRINGBYTESSTRUCTARRAYTIMESTAMPDATETIMEDATETIMEGEOGRAPHYJSONINTERVALRANGE

The field type

standardTableDefinition

The table definition if the type is TABLE

io.kestra.plugin.gcp.bigquery.models.StandardTableDefinition

clusteringarray

SubTypestring

The clustering configuration for this table. If null, the table is not clustered

rangePartitioning

The range partitioning configuration for this table. If null, the table is not range-partitioned

io.kestra.plugin.gcp.bigquery.models.RangePartitioning

fieldstring

The range partitioning field.

range

The range of range partitioning.

streamingBuffer

Information on the table's streaming buffer, if it exists. Null if no streaming buffer exists

com.google.cloud.bigquery.StandardTableDefinition-StreamingBuffer

estimatedBytesinteger

estimatedRowsinteger

oldestEntryTimeinteger

timePartitioning

The time partitioning configuration for this table. If null, the table is not time-partitioned

io.kestra.plugin.gcp.bigquery.models.TimePartitioning

expirationstring

The duration for which to keep the storage for a partition. When expired, the storage for the partition is reclaimed. If null, the partition does not expire

fieldstring

If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field

requirePartitionFilterbooleanstring

If set to true, queries over this table require a partition filter (that can be used for partition elimination) to be specified

typestring

Possible Values

DAYHOURMONTHYEAR

The time partitioning type

typestring

Possible Values

TABLEVIEWMATERIALIZED_VIEWEXTERNALMODEL

The table's type

viewDefinition

The view definition if the type is VIEW

io.kestra.plugin.gcp.bigquery.models.ViewDefinition

querystring

The query whose result is persisted

viewUserDefinedFunctionsarray

User defined functions that can be used by query. Null if not set

io.kestra.plugin.gcp.bigquery.models.UserDefinedFunction

contentstring

The content of the user defined function

If type is INLINE, this is a code blob. If type is FROM_URI, this is a Google Cloud Storage URI (e.g. gs://bucket/path).

typestring

Possible Values

INLINEFROM_URI

The type of user defined function.

Outputs

creationTime string

Formatdate-time

Creation time

datasetId string

Dataset ID

definition

Table definition

Definitions

io.kestra.plugin.gcp.bigquery.models.TableDefinition-Output

externalTableDefinition

io.kestra.plugin.gcp.bigquery.models.ExternalTableDefinition-Output

autodetectboolean

compressionstring

formatTypestring

Possible Values

CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORC

ignoreUnknownValuesboolean

maxBadRecordsinteger

sourceUrisarray

SubTypestring

materializedViewDefinition

io.kestra.plugin.gcp.bigquery.models.MaterializedViewDefinition-Output

enableRefreshboolean

lastRefreshDatestring

Formatdate-time

querystring

refreshIntervalstring

Formatduration

schema

io.kestra.plugin.gcp.bigquery.models.Schema-Output

fieldsarray

io.kestra.plugin.gcp.bigquery.models.Field-Output

descriptionstring

modestring

Possible Values

NULLABLEREQUIREDREPEATED

namestring

policyTags

subFieldsarray

typestring

Possible Values

BOOLINT64FLOAT64NUMERICBIGNUMERICSTRINGBYTESSTRUCTARRAYTIMESTAMPDATETIMEDATETIMEGEOGRAPHYJSONINTERVALRANGE

standardTableDefinition

io.kestra.plugin.gcp.bigquery.models.StandardTableDefinition-Output

clusteringarray

SubTypestring

rangePartitioning

io.kestra.plugin.gcp.bigquery.models.RangePartitioning-Output

fieldstring

range

streamingBuffer

com.google.cloud.bigquery.StandardTableDefinition-StreamingBuffer

estimatedBytesinteger

estimatedRowsinteger

oldestEntryTimeinteger

timePartitioning

io.kestra.plugin.gcp.bigquery.models.TimePartitioning-Output

expirationstring

Formatduration

fieldstring

requirePartitionFilterboolean

typestring

Possible Values

DAYHOURMONTHYEAR

typestring

Possible Values

TABLEVIEWMATERIALIZED_VIEWEXTERNALMODEL

viewDefinition

io.kestra.plugin.gcp.bigquery.models.ViewDefinition-Output

querystring

viewUserDefinedFunctionsarray

io.kestra.plugin.gcp.bigquery.models.UserDefinedFunction-Output

contentstring

typestring

Possible Values

INLINEFROM_URI

description string

Description

encryptionConfiguration

Encryption configuration

Definitions

io.kestra.plugin.gcp.bigquery.models.EncryptionConfiguration-Output

kmsKeyNamestring

etag string

Table etag

expirationTime string

Formatdate-time

Expiration time

If absent, the table persists indefinitely; expired tables are deleted

friendlyName string

Friendly name

generatedId string

Service-generated table id

labels object

SubTypestring

Labels

lastModifiedTime string

Formatdate-time

Last modified time

numBytes integer

Size in bytes

numLongTermBytes integer

Long-term storage bytes

Bytes billed at long-term storage rates

numRows integer

Row count

projectId string

Project ID

requirePartitionFilter boolean

Require partition filter

True when queries must specify a partition filter for partition elimination

selfLink string

Self link URL

API URL for get or update requests

table string

Table name

Liquibase

Tasks that run Liquibase change management commands.

Data

PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Data

Google Cloud UpdateTableCertifiedUpdate BigQuery table metadata

Google Cloud UpdateTable

More Plugins in this Category

Liquibase

PostgreSQL

Google Cloud UpdateTable
Certified
Update BigQuery table metadata