S2 Read
CertifiedRead records from an S2 stream
S2 Read
Read records from an S2 stream
Reads records from an S2 stream by sequence number, timestamp, or tail offset.
type: io.kestra.plugin.s2.ReadExamples
Read latest records
id: s2_read
namespace: company.team
tasks:
- id: read
type: io.kestra.plugin.s2.Read
accessToken: "{{ secret('S2_ACCESS_TOKEN') }}"
basin: my-basin
stream: orders
fetchType: FETCH
maxRecords: 100
Properties
accessToken *string
S2 access token
Access token used to authenticate with S2
basin *string
S2 basin
S2 basin containing the streams
stream *string
Stream
Name of the S2 stream to read records from.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150basinEndpoint string
https://{basin}.b.s2.dev/v1Basin endpoint
S2 basin API endpoint. The {basin} placeholder is replaced by the basin name. When the endpoint has no placeholder, the basin is sent in the S2-Basin header, which is how a custom endpoint such as s2-lite is addressed.
fetchType string
STORESTOREFETCHFETCH_ONENONEFetch type
Determines whether records are returned directly, a single record is returned, or records are stored in Kestra internal storage.
maxRecords integerstring
100Maximum records
Maximum number of records to read, between 1 and 1000.
seqNum integerstring
Start sequence number
Start from the first record whose sequence number is greater than or equal to this value.
tailOffset integerstring
Start tail offset
Start this many records before the tail of the stream.
timestamp integerstring
Start timestamp
Start from the first record whose timestamp is greater than or equal to this value.
Outputs
record
Record
The first record that was read. Only set when 'fetchType' is FETCH_ONE and a record exists.
io.kestra.plugin.s2.Read-Record
Record body
Body of the S2 record.
Record headers
Name value header pairs of the S2 record.
Sequence number
Sequence number assigned by S2.
Timestamp
Timestamp of the record.
records array
Records
The records that were read. Only set when 'fetchType' is FETCH.
io.kestra.plugin.s2.Read-Record
Record body
Body of the S2 record.
Record headers
Name value header pairs of the S2 record.
Sequence number
Sequence number assigned by S2.
Timestamp
Timestamp of the record.
size integer
Size
Number of records read.
tail integer
Tail
Sequence number that will be assigned to the next record on the stream.
uri string
uriRecords URI
URI of the ION file in Kestra internal storage containing the records. Only set when 'fetchType' is STORE.
