Serialization CsvToIon
CertifiedConvert a CSV file to the Amazon ION format
Serialization CsvToIon
Convert a CSV file to the Amazon ION format
Supports configurable field separator, text delimiter, charset, and header detection. The value \N is treated as null in any field. Use onBadLines to control error handling for malformed rows. A leading UTF-8 byte-order mark is stripped automatically. Trailing unnamed header columns (e.g. from a trailing field separator) are dropped by default; set onEmptyHeader to RENAME to keep them with generated names (col_0, col_1, ...) instead.
type: io.kestra.plugin.serdes.csv.CsvToIonExamples
Convert a CSV file to the Amazon ION format.
id: csv_to_ion
namespace: company.team
tasks:
- id: http_download
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/products.csv
- id: to_ion
type: io.kestra.plugin.serdes.csv.CsvToIon
from: "{{ outputs.http_download.uri }}"
Properties
from *Requiredstring
Source file URI
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
allowExtraCharsAfterClosingQuote booleanstring
falseAllow extra characters after a closing quote
charset string
UTF-8The name of a supported charset
fieldSeparator string
,The field separator character
header booleanstring
trueSpecifies if the first line should be the header
maxBufferSize integerstring
16777216Maximum CSV parser buffer size (bytes)
maxFieldSize integerstring
16777216Maximum field size (characters)
onBadLines string
ERRORERRORWARNSKIPHow to handle bad lines (e.g., a line with too many fields)
onEmptyHeader string
DROPDROPRENAMEHow to handle columns whose header name is empty
DROP (default) removes trailing unnamed columns. RENAME keeps every column and names unnamed ones col_0, col_1, ... to avoid losing data.
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
skipEmptyRows booleanstring
falseSpecifies if empty rows should be skipped
skipRows integerstring
0Number of lines to skip at the start of the file
textDelimiter string
"The text delimiter character
Outputs
size integer
0The number of records converted
uri string
uriURI of a temporary result file
Metrics
records counter
Number of records converted
