Google Cloud LaunchFlexTemplate
CertifiedLaunch a Flex Template Dataflow job
Google Cloud LaunchFlexTemplate
Launch a Flex Template Dataflow job
Starts a Dataflow pipeline defined as a Flex Template packaged in a Docker container image.
type: io.kestra.plugin.gcp.dataflow.LaunchFlexTemplateExamples
Launch a Flex Template job.
id: dataflow_flex_template
namespace: company.team
tasks:
- id: launch
type: io.kestra.plugin.gcp.dataflow.LaunchFlexTemplate
projectId: "{{ secret('GCP_PROJECT_ID') }}"
location: us-central1
jobName: my-flex-job
containerSpecGcsPath: gs://my-bucket/flex-templates/spec.json
parameters:
inputSubscription: projects/my-project/subscriptions/my-sub
outputTable: my-project:my_dataset.my_table
environment:
maxWorkers: 10
Properties
containerSpecGcsPath *string
The Cloud Storage path to the Flex Template container spec JSON file
jobName *string
The unique job name
location *string
The regional endpoint (e.g. us-central1)
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150environment object
Template runtime environment options
impersonatedServiceAccount string
The GCP service account to impersonate
parameters object
Template parameters
projectId string
The GCP project ID
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serviceAccount string
The GCP service account
Outputs
jobId string
The launched job ID
jobState string
The job state
