Google Cloud Bigtable ReadRows

Google Cloud Bigtable ReadRows
Certified
Read rows from a Google Cloud Bigtable table

Reads rows from a Bigtable table using an optional row key range, row key prefix, or cell filter. Supports FETCH (return the rows in the output, at most 5000 when limit is not set), FETCH_ONE (return only the first row), and STORE (persist all rows to internal storage as ion, recommended for large result sets) fetch types

yaml
type: io.kestra.plugin.gcp.bigtable.ReadRows

Examples

Read rows from Bigtable by row key prefix and store the result

yaml
id: bigtable_read_rows
namespace: company.team

tasks:
  - id: read
    type: io.kestra.plugin.gcp.bigtable.ReadRows
    projectId: "{{ secret('GCP_PROJECT_ID') }}"
    instanceId: my-instance
    tableId: sensor-readings
    rowKeyPrefix: "device-42#"
    fetchType: STORE

Read the first row within a row key range

yaml
id: bigtable_read_one_row
namespace: company.team

tasks:
  - id: read
    type: io.kestra.plugin.gcp.bigtable.ReadRows
    projectId: "{{ secret('GCP_PROJECT_ID') }}"
    instanceId: my-instance
    tableId: sensor-readings
    rowKeyStart: "device-42#001"
    rowKeyEnd: "device-42#002"
    fetchType: FETCH_ONE

Installation

Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:

kestractl plugins get io.kestra.plugin:plugin-gcp --compatible-for $KESTRA_VERSION

For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.

For Kestra CLI, replace LATEST with a specific version or version range to pin the plugin version.

Properties

instanceId *string

The Bigtable instance ID

An instance is a container for your tables within a given GCP project.

projectId *string

The GCP project ID

tableId *string

The Bigtable table ID to read from

assets

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions

io.kestra.core.models.assets.AssetsDeclaration

assetFailureBehaviorstring

Possible Values

IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

io.kestra.core.models.assets.AssetIdentifier

id*string

Min length1

typestring

outputs

The assets produced as outputs.

io.kestra.plugin.ee.assets.Dataset

id*string

Min length1

Max length150

type*object

descriptionstring

displayNamestring

metadataobject

Default{}

namespacestring

Min length1

Max length150

io.kestra.plugin.ee.assets.File

id*string

Min length1

Max length150

type*object

descriptionstring

displayNamestring

metadataobject

Default{}

namespacestring

Min length1

Max length150

io.kestra.plugin.ee.assets.Table

id*string

Min length1

Max length150

type*object

descriptionstring

displayNamestring

metadataobject

Default{}

namespacestring

Min length1

Max length150

io.kestra.plugin.ee.assets.VM

id*string

Min length1

Max length150

type*object

descriptionstring

displayNamestring

metadataobject

Default{}

namespacestring

Min length1

Max length150

io.kestra.core.models.assets.External

id*string

Min length1

Max length150

type*object

descriptionstring

displayNamestring

metadataobject

Default{}

namespacestring

Min length1

Max length150

io.kestra.core.models.assets.Custom

id*string

Min length1

Max length150

type*string

Min length1

Custom asset type

descriptionstring

displayNamestring

metadataobject

Default{}

namespacestring

Min length1

Max length150

columnFamily string

Only return cells from this column family

If not set, cells from all column families are returned

emulatorHost string

Bigtable emulator host (host: port), for local testing only

When set, the task connects to a local Bigtable emulator instead of the real Bigtable service, using plaintext transport and no credentials. Not intended for production use.

fetchType string

DefaultSTORE

Possible Values

STOREFETCHFETCH_ONENONE

Data storage method

FETCH_ONE outputs the first row, FETCH outputs the rows in the output variable (at most 5000 when limit is not set), STORE stores all rows to a file in internal storage and is recommended for large result sets

impersonatedServiceAccount string

The GCP service account to impersonate

limit integerstring

Maximum number of rows to read

If not set, all matching rows are read, except with fetchType FETCH, which returns at most 5000 rows and logs a warning when there are more

rowKeyEnd string

Exclusive end of the row key range to scan

Used together with rowKeyStart. Ignored if rowKeyPrefix is set

rowKeyPrefix string

Row key prefix to filter on

Mutually exclusive with rowKeyStart/rowKeyEnd

rowKeyStart string

Inclusive start of the row key range to scan

Mutually exclusive with rowKeyPrefix. If neither is set, the full table is scanned

scopes array

SubTypestring

The GCP scopes to be used

serviceAccount string

The GCP service account

Outputs

row object

The first matching row (only set when fetchType is FETCH_ONE)

Row keys, column qualifiers and values that are not valid UTF-8 are returned as base64. A matching rowKeyEncoding, qualifierEncoding or valueEncoding key is set to base64.

rowCount integer

Number of rows read

rows array

SubTypeobject

Rows read (only set when fetchType is FETCH)

Row keys, column qualifiers and values that are not valid UTF-8 are returned as base64. A matching rowKeyEncoding, qualifierEncoding or valueEncoding key is set to base64.