Google Cloud Trigger

Google Cloud Trigger
Certified
Wait for a Dataflow job to reach a target terminal state and trigger a flow execution

Polls the Dataflow service at a regular interval and triggers execution when a job matching the name prefix transitions to the target state. If multiple jobs match, the trigger returns the most recently completed one.

yaml
type: io.kestra.plugin.gcp.dataflow.Trigger

Examples

Trigger workflow when a Dataflow job completes.

yaml
id: dataflow_job_completion_trigger
namespace: company.team

triggers:
  - id: on_job_done
    type: io.kestra.plugin.gcp.dataflow.Trigger
    projectId: "{{ secret('GCP_PROJECT_ID') }}"
    location: us-central1
    jobNamePrefix: my-etl-
    targetState: JOB_STATE_DONE
    interval: PT2M

tasks:
  - id: notify
    type: io.kestra.plugin.core.log.Log
    message: "Dataflow job {{ trigger.jobId }} completed with state: {{ trigger.state }}"

Properties

jobNamePrefix *string

The Dataflow job name prefix to match

location *string

The regional endpoint (e.g. us-central1)

projectId *string

The GCP project ID

allowConcurrent boolean

Defaultfalse

Specifies whether a trigger is allowed to start a new execution even if a previous run is still in progress.

impersonatedServiceAccount string

The GCP service account to impersonate

interval string

DefaultPT1M

Formatduration

The interval between polls

lookback string

Lookback window applied to job state transitions

Defaults to the polling interval if not specified.

on string

DefaultCREATE_OR_UPDATE

Possible Values

CREATEUPDATECREATE_OR_UPDATE

Which state change events to trigger on

Can be CREATE, UPDATE or CREATE_OR_UPDATE.

scopes array

SubTypestring

The GCP scopes to be used

serviceAccount string

The GCP service account

stateKey string

State key

Override key used to persist trigger state; defaults to namespace/flow/id

stateTtl string

State TTL

Optional TTL for trigger state entries

stopAfter array

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

List of execution states after which a trigger should be stopped (a.k.a. disabled).

targetState string

DefaultJOB_STATE_DONE

Possible Values

JOB_STATE_UNKNOWNJOB_STATE_STOPPEDJOB_STATE_RUNNINGJOB_STATE_DONEJOB_STATE_FAILEDJOB_STATE_CANCELLEDJOB_STATE_UPDATEDJOB_STATE_DRAININGJOB_STATE_DRAINEDJOB_STATE_PENDINGJOB_STATE_CANCELLINGJOB_STATE_QUEUED

The target state to trigger on (e.g. JOB_STATE_DONE, JOB_STATE_FAILED)

when string

Defaulttrue

A condition that determines whether the trigger should run.

A Pebble expression evaluated at trigger time. The trigger fires only when the expression evaluates to a truthy value (true, a non-empty string, a non-zero number). Use this to gate trigger execution on dynamic runtime values such as execution labels, flow variables, or environment conditions.

Outputs

jobId string

The matched job ID

jobName string

The matched job name

state string

The terminal state matched