Serialization CsvToIon
CertifiedConvert a CSV file to the Amazon ION format
Serialization CsvToIon
Convert a CSV file to the Amazon ION format
Supports configurable field separator, text delimiter, charset, and header detection. The value \N is treated as null in any field. Use onBadLines to control error handling for malformed rows.
type: io.kestra.plugin.serdes.csv.CsvToIonExamples
Convert a CSV file to the Amazon ION format.
id: csv_to_ion
namespace: company.team
tasks:
- id: http_download
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/products.csv
- id: to_ion
type: io.kestra.plugin.serdes.csv.CsvToIon
from: "{{ outputs.http_download.uri }}"
Properties
from *Requiredstring
Source file URI
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
allowExtraCharsAfterClosingQuote booleanstring
falseAllow extra characters after a closing quote
charset string
UTF-8The name of a supported charset
fieldSeparator string
,The field separator character
header booleanstring
trueSpecifies if the first line should be the header
maxBufferSize integerstring
16777216Maximum CSV parser buffer size (bytes)
maxFieldSize integerstring
16777216Maximum field size (characters)
onBadLines string
ERRORERRORWARNSKIPHow to handle bad lines (e.g., a line with too many fields)
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
skipEmptyRows booleanstring
falseSpecifies if empty rows should be skipped
skipRows integerstring
0Number of lines to skip at the start of the file
textDelimiter string
"The text delimiter character
Outputs
size integer
0The number of records converted
uri string
uriURI of a temporary result file
Metrics
records counter
Number of records converted
