Run Multiple Python Versions Side by Side in One Flow
Run the same script on Python 3.9 and Python 3.13 in one flow, each task pinned to its own container image, to evaluate a new runtime without touching production.
Download a zip file, unzip it, and process a CSV file with Kestra. Automate end-to-end file ingestion and data processing in a single workflow.
This blueprint builds an end-to-end file processing pipeline in Kestra: it downloads a compressed ZIP archive over HTTP, decompresses it, reads the CSV inside, and writes the records to a schema-validated Parquet file. It solves the common ingestion problem of turning raw, zipped CSV exports into a typed, columnar format that is ready for analytics, data lakes, and downstream warehouse loads, without any glue scripts or manual steps.
get_zipfile (io.kestra.plugin.core.http.Download) fetches the Divvy bikeshare ZIP archive over HTTP. The URL is templated with the file_id variable so you can swap datasets by changing one value.unzip (io.kestra.plugin.compress.ArchiveDecompress) decompresses the archive using the ZIP algorithm and exposes the extracted files as outputs.csv_to_ion (io.kestra.plugin.serdes.csv.CsvToIon) reads the extracted CSV and converts it to Kestra's internal Ion format for efficient, streamable processing.to_parquet (io.kestra.plugin.serdes.avro.IonToAvro) writes the data to Parquet, enforcing an explicit Avro schema with typed fields (strings, doubles, nullable coordinates, and timestamp-millis columns) and a datetimeFormat for parsing dates.{{ outputs.get_zipfile.uri }} and friends).Kestra turns a chain of shell steps into a declarative YAML workflow with built-in retries, full execution lineage, and event-driven or scheduled triggers. The file utilities themselves (download, unzip, CSV and Parquet conversion) have no scheduler, no state, and no observability: Kestra supplies the orchestration layer, passing files between tasks through internal storage and giving you replays and per-task logs that standalone serdes tools cannot offer.
This flow uses no secrets. The source file is fetched from a public HTTP endpoint.
to_parquet output to confirm the schema-validated Parquet file.file_id variable to process a different dataset.schema to match your own CSV columns.Schedule trigger to run on a cadence, or a webhook trigger for event-driven ingestion.