Delete BigQuery partitions between intervals.
type: "io.kestra.plugin.gcp.bigquery.deletepartitions"id: gcp_bq_delete_partitions
namespace: company.team
tasks:
- id: delete_partitions
type: io.kestra.plugin.gcp.bigquery.DeletePartitions
projectId: my-project
dataset: my-dataset
table: my-table
partitionType: DAY
from: "{{ now() | dateAdd(-30, 'DAYS') }}"
to: "{{ now() | dateAdd(-7, 'DAYS') }}"
The dataset's user-defined ID.
The inclusive starting date or integer.
If the partition :
- is a numeric range, must be a valid integer
- is a date, must a valid datetime like
{{ now() }}
DAYHOURMONTHYEARRANGEThe partition type of the table
The table's user-defined ID.
The inclusive ending date or integer.
If the partition :
- is a numeric range, must be a valid integer
- is a date, must a valid datetime like
{{ now() }}
The GCP service account to impersonate.
The geographic location where the dataset should reside.
This property is experimental and might be subject to change or removed.
See Dataset Location
The GCP project ID.
Automatic retry for retryable BigQuery exceptions.
Some exceptions (especially rate limit) are not retried by default by BigQuery client, we use by default a transparent retry (not the kestra one) to handle this case. The default values are exponential of 5 seconds for a maximum of 15 minutes and ten attempts
["due to concurrent update","Retrying the job may solve the problem","Retrying may solve the problem"]The messages which would trigger an automatic retry.
Message is tested as a substring of the full message, and is case insensitive.
["rateLimitExceeded","jobBackendError","backendError","internalError","jobInternalError"]The reasons which would trigger an automatic retry.
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used.
The GCP service account.
The dataset's id
Partitions deleted
The project's id
The table name
The number of partitions deleted.
durationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION >= 1durationfalsedurationdurationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION >= 1durationfalsedurationdurationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION >= 1durationfalse