Google Cloud RunTransferConfig
CertifiedTrigger a BigQuery Data Transfer Service run
Google Cloud RunTransferConfig
Trigger a BigQuery Data Transfer Service run
Starts a manual run of an existing BigQuery Data Transfer Service transfer config, or re-attaches to a run that is already pending or running for that config so a retried execution never starts a duplicate run. Optionally polls the run until it reaches a terminal state, on a wall-clock deadline.
type: io.kestra.plugin.gcp.bigquery.RunTransferConfigExamples
Trigger a scheduled query transfer config and wait for it to complete.
id: bigquery_data_transfer
namespace: company.team
tasks:
- id: trigger_transfer
type: io.kestra.plugin.gcp.bigquery.RunTransferConfig
projectId: "{{ secret('GCP_PROJECT_ID') }}"
transferConfigName: projects/my-project/locations/us/transferConfigs/615123456789012345
pollInterval: PT15S
maxDuration: PT1H
# Enterprise Edition: emit the destination as a data-lineage asset.
assets:
enableAuto: true
Properties
transferConfigName *string
The transfer config resource name
Format: projects/{project}/locations/{location}/transferConfigs/{config}.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150impersonatedServiceAccount string
The GCP service account to impersonate
maxDuration string
PT1HThe maximum duration to wait before timing out, used only when wait is true
pollInterval string
PT15SThe interval between polls, used only when wait is true
projectId string
The GCP project ID
reattach booleanstring
trueWhether to re-attach to an already pending or running run of this config instead of starting a new one
When true (default), the task first looks for a run of this config that is already pending or running and adopts it, so a retried execution never starts a duplicate run. A pending run whose schedule time is in the future is never adopted: it is queued, not in-flight. A running run is always in-flight and stays eligible whatever its schedule time. This matters for transfers with a refresh window (Google Ads, GA4, YouTube), where Google permanently keeps a chain of pending runs queued ahead of now, one per backfill date. By default any in-flight run is adopted regardless of its age; set reattachMaxAge to opt into treating an old in-flight run as stale so a fresh run is started instead. Set to false to always start a new run.
reattachMaxAge string
The maximum age of an in-flight run that can still be re-attached to
Used only when reattach is true. When set, an in-flight (pending or running) run whose schedule time is older than now - reattachMaxAge is treated as stale and is NOT adopted -- a fresh run is started instead. When unset (default), any in-flight run is adopted regardless of its age. This is independent of maxDuration, which only bounds how long this task waits for a run to reach a terminal state.
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serviceAccount string
The GCP service account
wait booleanstring
trueWhether to wait until the transfer run reaches a terminal state
When false, the task returns as soon as the run is triggered and does not emit a destination asset, since completion is not confirmed.
Outputs
destinationDatasetId string
The destination dataset ID of the transfer config
location string
The location, parsed from the transfer config resource name
projectId string
The GCP project ID, parsed from the transfer config resource name
reattached boolean
falseWhether an existing in-flight run was re-attached instead of starting a new one
runName string
The full resource name of the transfer run
state string
The state of the transfer run, terminal when wait is true
Metrics
duration timer
The wall-clock time spent waiting for the transfer run to reach a terminal state, recorded only when wait is true.
reattached counter
Whether the task re-attached to an already in-flight transfer run (1) or started a new one (0).
