Google Cloud Copy

Google Cloud Copy

Certified

Copy or snapshot BigQuery tables

Runs a table copy job between tables or partitions. Supports COPY, SNAPSHOT, RESTORE, or CLONE operations and honors create/write dispositions.

yaml
type: io.kestra.plugin.gcp.bigquery.Copy
yaml
id: gcp_bq_copy
namespace: company.team

tasks:
  - id: copy
    type: io.kestra.plugin.gcp.bigquery.Copy
    operationType: COPY
    sourceTables:
      - "my-project.my_dataset.my_table$20130908"
    destinationTable: "my-project.my_dataset.my_table"
Properties

Destination table

Target table for the operation; must match the selected operation type

SubTypestring

Source tables

Tables or partitions to copy; accepts partition decorators

Possible Values
CREATE_IF_NEEDEDCREATE_NEVER

Create disposition

Whether the job may create the destination table

Defaultfalse

Dry run

If true, validates the job and returns statistics without running it

The GCP service account to impersonate

Job timeout

Optional max duration; BigQuery may terminate the job when exceeded

Job labels

Dataset location

Optional BigQuery location for created or targeted resources. Experimental and may change; see BigQuery dataset location documentation.

Possible Values
COPYSNAPSHOTRESTORECLONE

Copy operation type

COPY (default), SNAPSHOT, RESTORE, or CLONE

Reference (ref) of the pluginDefaults to apply to this task.

The GCP project ID

Automatic BigQuery retry policy

Optional custom retry policy for retryable BigQuery errors. If unset, uses an exponential backoff starting at 5s (per-attempt interval capped at 60m), with a total duration of up to 15m and a maximum of 10 attempts.

Definitions

Retry with a fixed delay between attempts.

interval*Requiredstring
Formatduration
type*Requiredobject
behaviorstring
DefaultRETRY_FAILED_TASK
Possible Values
RETRY_FAILED_TASKCREATE_NEW_EXECUTION
maxAttemptsinteger
Minimum>= 1
maxDurationstring
Formatduration
warningOnRetryboolean
Defaultfalse

Retry with exponentially increasing delays between attempts.

interval*Requiredstring
Formatduration
maxInterval*Requiredstring
Formatduration
type*Requiredobject
behaviorstring
DefaultRETRY_FAILED_TASK
Possible Values
RETRY_FAILED_TASKCREATE_NEW_EXECUTION
delayFactornumber
maxAttemptsinteger
Minimum>= 1
maxDurationstring
Formatduration
warningOnRetryboolean
Defaultfalse

Retry with a random delay within a configurable range between attempts.

maxInterval*Requiredstring
Formatduration
minInterval*Requiredstring
Formatduration
type*Requiredobject
behaviorstring
DefaultRETRY_FAILED_TASK
Possible Values
RETRY_FAILED_TASKCREATE_NEW_EXECUTION
maxAttemptsinteger
Minimum>= 1
maxDurationstring
Formatduration
warningOnRetryboolean
Defaultfalse
SubTypestring
Default["due to concurrent update","Retrying the job may solve the problem","Retrying may solve the problem"]

Retry message substrings

Case-insensitive substrings that, if found in the error message, trigger an automatic retry

SubTypestring
Default["rateLimitExceeded","jobBackendError","backendError","internalError","jobInternalError"]

Retry reasons

BigQuery error reasons that trigger an automatic retry; evaluated against error reason strings

SubTypestring
Default["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used

The GCP service account

Possible Values
WRITE_TRUNCATEWRITE_TRUNCATE_DATAWRITE_APPENDWRITE_EMPTY

Write disposition

Action when destination exists

Job ID

The time it took for the job to run.

The number of child jobs executed.

More Plugins in this Category

Liquibase
Liquibase

Tasks that run Liquibase change management commands.

Data

2 tasks

6 blueprints

OPA (Open Policy Agent)
OPA (Open Policy Agent)
Enterprise

Tasks to manage Open Policy Agent policies and decisions, including upload, evaluation, and partial evaluation.

Infrastructure

5 tasks

3 blueprints