CopyPartitions
Copy partitions between interval to another table
type: "io.kestra.plugin.gcp.bigquery.CopyPartitions"id: gcp_bq_copy_partitions
namespace: company.team
tasks:
- id: copy_partitions
type: io.kestra.plugin.gcp.bigquery.CopyPartitions
projectId: my-project
dataset: my-dataset
table: my-table
destinationTable: my-dest-table
partitionType: DAY
from: "{{ now() | dateAdd(-30, 'DAYS') }}"
to: "{{ now() | dateAdd(-7, 'DAYS') }}"
YESThe dataset's user-defined ID.
YESThe inclusive starting date or integer.
If the partition :
- is a numeric range, must be a valid integer
- is a date, must a valid datetime like
{{ now() }}
YESDAYHOURMONTHYEARRANGEThe partition type of the table
YESThe table's user-defined ID.
YESThe inclusive ending date or integer.
If the partition :
- is a numeric range, must be a valid integer
- is a date, must a valid datetime like
{{ now() }}
YESCREATE_IF_NEEDEDCREATE_NEVERWhether the job is allowed to create tables.
YESThe table where to put query results.
If not provided, a new table is created.
YESfalseYESThe GCP service account to impersonate.
YESdurationJob timeout.
If this time limit is exceeded, BigQuery may attempt to terminate the job.
YESThe labels associated with this job.
You can use these to organize and group your jobs. Label keys and values can be no longer than 63 characters, can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. Label values are optional. Label keys must start with a letter and each label in the list must have a different key.
YESThe geographic location where the dataset should reside.
This property is experimental and might be subject to change or removed.
See Dataset Location
YESThe GCP project ID.
NOYES["due to concurrent update","Retrying the job may solve the problem"]The messages which would trigger an automatic retry.
Message is tested as a substring of the full message, and is case insensitive.
YES["rateLimitExceeded","jobBackendError","internalError","jobInternalError"]The reasons which would trigger an automatic retry.
YES["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used.
YESThe GCP service account.
YESWRITE_TRUNCATEWRITE_APPENDWRITE_EMPTYThe action that should occur if the destination table already exists.
The dataset's id
The job id
Partitions copied
The project's id
The table name
NOdurationNORETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTIONNO >= 1NOdurationNOconstantNOfalseNOdurationNOdurationNORETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTIONNO >= 1NOdurationNOrandomNOfalseNOdurationNOdurationNORETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTIONNONO >= 1NOdurationNOexponentialNOfalse