Update a BigQuery table’s metadata.

yaml
type: "io.kestra.plugin.gcp.bigquery.updatetable"

Examples

yaml
id: gcp_bq_update_table
namespace: company.team

tasks:
  - id: update_table
    type: io.kestra.plugin.gcp.bigquery.UpdateTable
    projectId: my-project
    dataset: my-dataset
    table: my-table
    expirationDuration: PT48H

Properties

dataset*string

The dataset's user-defined ID.

table*string

The table's user-defined ID.

descriptionstring

encryptionConfiguration

Definitions

io.kestra.plugin.gcp.bigquery.models.EncryptionConfiguration

The KMS key name.string

expirationDurationstring

Formatduration

Sets the duration, since now, when this table expires.

friendlyNamestring

impersonatedServiceAccountstring

The GCP service account to impersonate.

labelsobject

SubTypestring

locationstring

The geographic location where the dataset should reside.

This property is experimental and might be subject to change or removed.

See Dataset Location

projectIdstring

The GCP project ID.

requirePartitionFilterbooleanstring

retryAuto

Automatic retry for retryable BigQuery exceptions.

Some exceptions (especially rate limit) are not retried by default by BigQuery client, we use by default a transparent retry (not the kestra one) to handle this case. The default values are exponential of 5 seconds for a maximum of 15 minutes and ten attempts

Definitions

io.kestra.core.models.tasks.retrys.Constant

interval*string

Formatduration

type*object

behaviorstring

DefaultRETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

maxAttemptsinteger

Minimum>= 1

maxDurationstring

Formatduration

warningOnRetryboolean

Defaultfalse

io.kestra.core.models.tasks.retrys.Exponential

interval*string

Formatduration

maxInterval*string

Formatduration

type*object

behaviorstring

DefaultRETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

delayFactornumber

maxAttemptsinteger

Minimum>= 1

maxDurationstring

Formatduration

warningOnRetryboolean

Defaultfalse

io.kestra.core.models.tasks.retrys.Random

maxInterval*string

Formatduration

minInterval*string

Formatduration

type*object

behaviorstring

DefaultRETRY_FAILED_TASK

Possible Values

RETRY_FAILED_TASKCREATE_NEW_EXECUTION

maxAttemptsinteger

Minimum>= 1

maxDurationstring

Formatduration

warningOnRetryboolean

Defaultfalse

retryMessagesarray

SubTypestring

Default["due to concurrent update","Retrying the job may solve the problem","Retrying may solve the problem"]

The messages which would trigger an automatic retry.

Message is tested as a substring of the full message, and is case insensitive.

retryReasonsarray

SubTypestring

Default["rateLimitExceeded","jobBackendError","backendError","internalError","jobInternalError"]

The reasons which would trigger an automatic retry.

scopesarray

SubTypestring

Default["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used.

serviceAccountstring

The GCP service account.

tableDefinition

Definitions

io.kestra.plugin.gcp.bigquery.models.TableDefinition

externalTableDefinition

The external table definition if the type is EXTERNAL.

io.kestra.plugin.gcp.bigquery.models.ExternalTableDefinition

autodetectbooleanstring

Whether automatic detection of schema and format options should be performed.

compressionstring

The compression type of the data source.

formatTypestring

Possible Values

CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORC

The source format, and possibly some parsing options, of the external data.

ignoreUnknownValuesbooleanstring

Whether BigQuery should allow extra values that are not represented in the table schema.

If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result.

maxBadRecordsintegerstring

The maximum number of bad records that BigQuery can ignore when reading data.

If the number of bad records exceeds this value, an invalid error is returned in the job result.

sourceUrisarray

SubTypestring

The fully-qualified URIs that point to your data in Google Cloud Storage.

Each URI can

contain one '*' wildcard character that must come after the bucket's name. Size limits related
to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size
across all URIs.

materializedViewDefinition

The materialized view definition if the type is MATERIALIZED_VIEW.

io.kestra.plugin.gcp.bigquery.models.MaterializedViewDefinition

enableRefreshbooleanstring

Whether automatic refresh is enabled for the materialized view when the base table is updated.

lastRefreshDatestring

Formatdate-time

Date when this materialized view was last modified.

querystring

refreshIntervalstring

Formatduration

The maximum frequency at which this materialized view will be refreshed.

schema

The table's schema.

io.kestra.plugin.gcp.bigquery.models.Schema

The fields associated with this schema.array

io.kestra.plugin.gcp.bigquery.models.Field

descriptionstring

The field description.

modestring

Possible Values

NULLABLEREQUIREDREPEATED

The field mode.

By default, Field.Mode.NULLABLE is used.

namestring

The field name.

policyTags

The policy tags for the field.

subFieldsarray

The list of sub-fields if type is a LegacySQLType.RECORD. Returns null otherwise.

typestring

Possible Values

BOOLINT64FLOAT64NUMERICBIGNUMERICSTRINGBYTESSTRUCTARRAYTIMESTAMPDATETIMEDATETIMEGEOGRAPHYJSONINTERVALRANGE

The field type.

standardTableDefinition

The table definition if the type is TABLE.

io.kestra.plugin.gcp.bigquery.models.StandardTableDefinition

clusteringarray

SubTypestring

Returns the clustering configuration for this table. If {@code null}, the table is not clustered.

rangePartitioning

Returns the range partitioning configuration for this table. If {@code null}, the table is not range-partitioned.

io.kestra.plugin.gcp.bigquery.models.RangePartitioning

The range of range partitioning.

The range partitioning field.string

streamingBuffer

Returns information on the table's streaming buffer, if exists. Returns {@code null} if no streaming buffer exists.

com.google.cloud.bigquery.StandardTableDefinition-StreamingBuffer

estimatedBytesinteger

estimatedRowsinteger

oldestEntryTimeinteger

timePartitioning

Returns the time partitioning configuration for this table. If {@code null}, the table is not time-partitioned.

io.kestra.plugin.gcp.bigquery.models.TimePartitioning

expirationstring

Formatduration

The number of milliseconds for which to keep the storage for a partition. When expired, the storage for the partition is reclaimed. If null, the partition does not expire.

fieldstring

If not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field.

requirePartitionFilterbooleanstring

If set to true, queries over this table require a partition filter (that can be used for partition elimination) to be specified.

typestring

Possible Values

DAYHOURMONTHYEAR

The time partitioning type.

typestring

Possible Values

TABLEVIEWMATERIALIZED_VIEWEXTERNALMODEL

The table's type.

viewDefinition

The view definition if the type is VIEW.

io.kestra.plugin.gcp.bigquery.models.ViewDefinition

querystring

The query whose result is persisted.

viewUserDefinedFunctionsarray

User defined functions that can be used by query. Returns {@code null} if not set.

io.kestra.plugin.gcp.bigquery.models.UserDefinedFunction

The type of user defined function.string

Possible Values

INLINEFROM_URI

Type of UserDefinedFunctionstring

If type is UserDefinedFunction.Type.INLINE, this method returns a code blob. If type is UserDefinedFunction.Type.FROM_URI, the method returns a Google Cloud Storage URI (e.g. gs://bucket/path).

Outputs

creationTimestring

Formatdate-time

The time when this table was created.

datasetIdstring

The dataset's ID.

definition

The table definition.

Definitions

io.kestra.plugin.gcp.bigquery.models.TableDefinition-Output

externalTableDefinition

io.kestra.plugin.gcp.bigquery.models.ExternalTableDefinition-Output

autodetectboolean

compressionstring

formatTypestring

Possible Values

CSVJSONBIGTABLEDATASTORE_BACKUPAVROGOOGLE_SHEETSPARQUETORC

ignoreUnknownValuesboolean

maxBadRecordsinteger

sourceUrisarray

SubTypestring

materializedViewDefinition

io.kestra.plugin.gcp.bigquery.models.MaterializedViewDefinition-Output

enableRefreshboolean

lastRefreshDatestring

Formatdate-time

querystring

refreshIntervalstring

Formatduration

schema

io.kestra.plugin.gcp.bigquery.models.Schema-Output

fieldsarray

io.kestra.plugin.gcp.bigquery.models.Field-Output

descriptionstring

modestring

Possible Values

NULLABLEREQUIREDREPEATED

namestring

policyTags

subFieldsarray

typestring

Possible Values

BOOLINT64FLOAT64NUMERICBIGNUMERICSTRINGBYTESSTRUCTARRAYTIMESTAMPDATETIMEDATETIMEGEOGRAPHYJSONINTERVALRANGE

standardTableDefinition

io.kestra.plugin.gcp.bigquery.models.StandardTableDefinition-Output

clusteringarray

SubTypestring

rangePartitioning

io.kestra.plugin.gcp.bigquery.models.RangePartitioning-Output

fieldstring

range

streamingBuffer

com.google.cloud.bigquery.StandardTableDefinition-StreamingBuffer

estimatedBytesinteger

estimatedRowsinteger

oldestEntryTimeinteger

timePartitioning

io.kestra.plugin.gcp.bigquery.models.TimePartitioning-Output

expirationstring

Formatduration

fieldstring

requirePartitionFilterboolean

typestring

Possible Values

DAYHOURMONTHYEAR

typestring

Possible Values

TABLEVIEWMATERIALIZED_VIEWEXTERNALMODEL

viewDefinition

io.kestra.plugin.gcp.bigquery.models.ViewDefinition-Output

querystring

viewUserDefinedFunctionsarray

io.kestra.plugin.gcp.bigquery.models.UserDefinedFunction-Output

contentstring

typestring

Possible Values

INLINEFROM_URI

descriptionstring

The user-friendly description for the table.

encryptionConfiguration

The encryption configuration.

Definitions

io.kestra.plugin.gcp.bigquery.models.EncryptionConfiguration-Output

kmsKeyNamestring

etagstring

The hash of the table resource.

expirationTimestring

Formatdate-time

Returns the time when this table expires.

If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

friendlyNamestring

The user-friendly name for the table.

generatedIdstring

The service-generated id for the table.

labelsobject

SubTypestring

Return a map for labels applied to the table.

lastModifiedTimestring

Formatdate-time

The time when this table was last modified.

numBytesinteger

The size of this table in bytes.

numLongTermBytesinteger

The number of bytes considered "long-term storage" for reduced billing purposes.

numRowsinteger

The number of rows of data in this table.

projectIdstring

The project's ID.

requirePartitionFilterboolean

Return true if a partition filter (that can be used for partition elimination) is required for queries over this table.

selfLinkstring

The URL that can be used to access the resource again. The returned URL can be used for get or update requests.

tablestring

The table name.

Core

CoreStorageScript

Resend

Plugin Resend for Kestra

Messaging

MQTT

This sub-group of plugins contains tasks for using a MQTT message broker.

Messaging

More Plugins in this Category

Core

Resend

MQTT

1.2.0