Google Cloud Consume
CertifiedConsume messages from Pub/Sub
Google Cloud Consume
Consume messages from Pub/Sub
Pulls messages from a subscription until maxRecords or maxDuration is reached. Writes messages to a Kestra storage file and acks them.
type: io.kestra.plugin.gcp.pubsub.ConsumeExamples
id: gcp_pubsub_consume
namespace: company.team
tasks:
- id: consume
type: io.kestra.plugin.gcp.pubsub.Consume
topic: topic-test
maxRecords: 10
projectId: "{{ secret('GCP_PROJECT_ID') }}"
subscription: my-subscription
Properties
subscription *string
Subscription
Subscription name; auto-created when autoCreateSubscription is true
topic *string
Topic name
Pub/Sub topic ID (without project prefix)
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150autoCreateSubscription booleanstring
trueAuto-create subscription
Create the subscription if missing; default true
impersonatedServiceAccount string
The GCP service account to impersonate
maxDuration string
Max duration
Duration limit (ISO-8601); stop when reached
maxRecords integerstring
Max records
Stop after this many messages
projectId string
The GCP project ID
scopes array
["https://www.googleapis.com/auth/cloud-platform"]The GCP scopes to be used
serdeType string
STRINGSTRINGJSONSerde type
Serializer/deserializer for message payloads; defaults to STRING
serviceAccount string
The GCP service account
Outputs
count integer
Consumed message count
uri string
uriMessages file URI
Kestra storage URI of the file containing consumed messages
Metrics
records counter
recordsNumber of records consumed from the Pub/Sub topic.
