Google Cloud ExtractToGcs
CertifiedExport BigQuery table to GCS
Google Cloud ExtractToGcs
Export BigQuery table to GCS
Runs an extract job from a table or partition to one or more GCS URIs. Supports CSV/JSON/AVRO, optional compression, custom delimiter, and AVRO logical types. Prints a header row by default. The job id is derived from the taskrun, so a worker-loss resubmit adopts the job the lost worker started instead of running it twice. A retry after a failed job submits under the next id in a deterministic chain (kestra_<taskRunId>_1, _2, ...).
type: io.kestra.plugin.gcp.bigquery.ExtractToGcsExamples
Extract a BigQuery table to a GCS bucket
id: gcp_bq_extract_to_gcs
namespace: company.team
tasks:
- id: extract_to_gcs
type: io.kestra.plugin.gcp.bigquery.ExtractToGcs
destinationUris:
- "gs://bucket_name/filename.csv"
sourceTable: "my-project.my_dataset.my_table"
format: CSV
fieldDelimiter: ';'
printHeader: true
Properties
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150compression string
Compression
Optional compression; if unset, files are uncompressed
destinationUris array
Destination URIs
One or more gs:// URIs to receive the export; wildcards allowed
fieldDelimiter string
Field delimiter
Delimiter between fields; defaults to comma
format string
Export format
CSV by default; AVRO and other formats supported
impersonatedServiceAccount string
The GCP service account to impersonate
jobTimeoutMs integerstring
Job timeout (ms)
Optional timeout; BigQuery may terminate the job if exceeded
labels object
Job labels
location string
Dataset location
Optional BigQuery location for created or targeted resources. Experimental and may change; see BigQuery dataset location documentation.
printHeader booleanstring
Print header
If true (default), include a header row in the export
projectId string
The GCP project ID
retryAuto
Automatic BigQuery retry policy
Optional custom retry policy for retryable BigQuery errors. If unset, uses an exponential backoff starting at 5s (per-attempt interval capped at 60m), with a total duration of up to 15m and a maximum of 10 attempts.
Constant retry
Retry with a fixed delay between attempts.
durationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION>= 1durationfalseExponential retry
Retry with exponentially increasing delays between attempts.
durationdurationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION>= 1durationfalseRandom retry
Retry with a random delay within a configurable range between attempts.
durationdurationRETRY_FAILED_TASKRETRY_FAILED_TASKCREATE_NEW_EXECUTION>= 1durationfalseretryMessages array
["due to concurrent update","Retrying the job may solve the problem","Retrying may solve the problem"]Retry message substrings
Case-insensitive substrings that, if found in the error message, trigger an automatic retry
retryReasons array
["rateLimitExceeded","jobBackendError","backendError","internalError","jobInternalError"]Retry reasons
BigQuery error reasons that trigger an automatic retry; evaluated against error reason strings
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serviceAccount string
The GCP service account
sourceTable string
Source table
Fully-qualified table (optionally partition) to export
useAvroLogicalTypes booleanstring
Use AVRO logical types
For AVRO exports, map supported columns to AVRO logical types
Outputs
destinationUris array
Destination URIs
fileCounts array
Number of extracted files
jobId string
Job ID
sourceTable string
Source table
Metrics
duration timer
The time it took for the job to run.
output.file_counts counter
The number of files extracted to GCS.
