Google Cloud RunTransferConfig
CertifiedTrigger a BigQuery Data Transfer Service run
Google Cloud RunTransferConfig
Trigger a BigQuery Data Transfer Service run
Starts a manual run of an existing BigQuery Data Transfer Service transfer config, or re-attaches to a run that is already pending or running for that config so a retried execution never starts a duplicate run. Optionally polls the run until it reaches a terminal state, on a wall-clock deadline.
type: io.kestra.plugin.gcp.bigquery.RunTransferConfigExamples
Trigger a scheduled query transfer config and wait for it to complete.
id: bigquery_data_transfer
namespace: company.team
tasks:
- id: trigger_transfer
type: io.kestra.plugin.gcp.bigquery.RunTransferConfig
projectId: "{{ secret('GCP_PROJECT_ID') }}"
transferConfigName: projects/my-project/locations/us/transferConfigs/615123456789012345
pollInterval: PT15S
maxDuration: PT1H
# Enterprise Edition: emit the destination as a data-lineage asset.
assets:
enableAuto: true
Properties
transferConfigName *Requiredstring
The transfer config resource name
Format: projects/{project}/locations/{location}/transferConfigs/{config}.
impersonatedServiceAccount string
The GCP service account to impersonate
maxDuration string
PT1HThe maximum duration to wait before timing out, used only when wait is true
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
pollInterval string
PT15SThe interval between polls, used only when wait is true
projectId string
The GCP project ID
reattach booleanstring
trueWhether to re-attach to an already pending or running run of this config instead of starting a new one
When true (default), the task first looks for a run of this config that is already pending or running and adopts it, so a retried execution never starts a duplicate run. By default any in-flight run is adopted regardless of its age; set reattachMaxAge to opt into treating an old in-flight run as stale so a fresh run is started instead. Set to false to always start a new run.
reattachMaxAge string
The maximum age of an in-flight run that can still be re-attached to
Used only when reattach is true. When set, an in-flight (pending or running) run whose schedule time is older than now - reattachMaxAge is treated as stale and is NOT adopted -- a fresh run is started instead. When unset (default), any in-flight run is adopted regardless of its age. This is independent of maxDuration, which only bounds how long this task waits for a run to reach a terminal state.
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serviceAccount string
The GCP service account
wait booleanstring
trueWhether to wait until the transfer run reaches a terminal state
When false, the task returns as soon as the run is triggered and does not emit a destination asset, since completion is not confirmed.
Outputs
destinationDatasetId string
The destination dataset ID of the transfer config
location string
The location, parsed from the transfer config resource name
projectId string
The GCP project ID, parsed from the transfer config resource name
reattached boolean
falseWhether an existing in-flight run was re-attached instead of starting a new one
runName string
The full resource name of the transfer run
state string
The state of the transfer run, terminal when wait is true
Metrics
duration timer
The wall-clock time spent waiting for the transfer run to reach a terminal state, recorded only when wait is true.
reattached counter
Whether the task re-attached to an already in-flight transfer run (1) or started a new one (0).
