ProtobufToIon
The plugin reads one or more Protobuf messages from a binary file or stream,
decodes them using a provided descriptor and message type name, and then serializes
the result as Ion data based on ProtoJSON Format.
It requires the following information regarding the Protobuf message:
- A descriptor file (
.desc), generated with --descriptor_set_out, that contains
the compiled Protobuf message definitions. - A type name (e.g.,
com.company.Product) corresponding to the root message to decode.
Here's an example of how to generate a descriptor file from a Protobuf file using protoc:
textprotoc --proto_path=src/main/proto --descriptor_set_out=products.desc src/main/proto/products.proto
The plugin reads one or more Protobuf messages from a binary file or stream, decodes them using a provided descriptor and message type name, and then serializes the result as Ion data based on ProtoJSON Format. It requires the following information regarding the Protobuf message:
- A descriptor file (
.desc), generated with--descriptor_set_out, that contains the compiled Protobuf message definitions. - A type name (e.g.,
com.company.Product) corresponding to the root message to decode.
Here's an example of how to generate a descriptor file from a Protobuf file using protoc:
protoc --proto_path=src/main/proto --descriptor_set_out=products.desc src/main/proto/products.proto
Convert a Protobuf file into Amazon Ion.
The plugin reads one or more Protobuf messages from a binary file or stream, decodes them using a provided descriptor and message type name, and then serializes the result as Ion data based on ProtoJSON Format. It requires the following information regarding the Protobuf message:
- A descriptor file (
.desc), generated with--descriptor_set_out, that contains the compiled Protobuf message definitions. - A type name (e.g.,
com.company.Product) corresponding to the root message to decode.
Here's an example of how to generate a descriptor file from a Protobuf file using protoc:
protoc --proto_path=src/main/proto --descriptor_set_out=products.desc src/main/proto/products.proto
type: "io.kestra.plugin.serdes.protobuf.ProtobufToIon"Examples
Convert a Protobuf file to the Amazon Ion format.
id: protobuf_to_ion
namespace: company.team
tasks:
- id: http_download
type: io.kestra.plugin.core.http.Download
uri: https://example.com/data/products.pb
- id: to_ion
type: io.kestra.plugin.serdes.protobuf.ProtobufToIon
from: "{{ outputs.http_download.uri }}"
descriptorFile: "kestra:///path/to/proto.desc"
typeName: com.company.Product
Properties
descriptorFile*Requiredstring
Protobuf Descriptor File
Protobuf descriptor file containing message definitions.
from*Requiredstring
Source file URI
The URI of the input Protobuf file to read.
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
typeName*Requiredstring
Fully qualified Protobuf message type name
For example: com.company.Product.
delimitedbooleanstring
falseIs the input a stream of length-delimited messages?
If true, the input file is expected to contain multiple length-delimited Protobuf messages. If false, it will be parsed as a single Protobuf message.
errorOnUnknownFieldsbooleanstring
falseWhether to error on unknown fields
If true, an error will be thrown if the input contains unknown fields.
Outputs
uristring
uriURI of a temporary result file
Metrics
recordscounter
Number of Protobuf messages converted
