Consume

type: "io.kestra.plugin.gcp.pubsub.Consume"

Consume messages from a Pub/Sub topic.

Requires a maxDuration or a maxRecords.

Examples

id: gcp_pubsub_consume
namespace: company.team

tasks:
  - id: consume
    type: io.kestra.plugin.gcp.pubsub.Consume
    topic: topic-test
    maxRecords: 10
    projectId: {{ secret('GCP_PROJECT_ID') }}
    subscription: my-subscription

Properties

serdeType

• Type: string
• Dynamic: ❌
• Required: ✔️
• Default: STRING
• Possible Values:
  • STRING
  • JSON

The serializer/deserializer to use.

subscription

• Type: string
• Dynamic: ✔️
• Required: ✔️

The Pub/Sub subscription.

The Pub/Sub subscription. It will be created automatically if it didn't exist and 'autoCreateSubscription' is enabled.

topic

• Type: string
• Dynamic: ✔️
• Required: ✔️

The Pub/Sub topic

The Pub/Sub topic. It must be created before executing the task.

autoCreateSubscription

• Type: boolean
• Dynamic: ❌
• Required: ❌
• Default: true

Whether the Pub/Sub subscription should be created if not exists.

impersonatedServiceAccount

• Type: string
• Dynamic: ✔️
• Required: ❌

The GCP service account to impersonate.

maxDuration

• Type: string
• Dynamic: ❌
• Required: ❌
• Format: duration

Max duration in the Duration ISO format, after that the task will end.

maxRecords

• Type: integer
• Dynamic: ❌
• Required: ❌

Max number of records, when reached the task will end.

projectId

• Type: string
• Dynamic: ✔️
• Required: ❌

The GCP project ID.

scopes

• Type: array
• SubType: string
• Dynamic: ✔️
• Required: ❌
• Default: [https://www.googleapis.com/auth/cloud-platform]

The GCP scopes to be used.

serviceAccount

• Type: string
• Dynamic: ✔️
• Required: ❌

The GCP service account.

Outputs

count

• Type: integer
• Required: ❌

Number of consumed rows.

uri

• Type: string
• Required: ❌
• Format: uri

File URI containing consumed messages.