Download a ZIP file, unzip it, and convert it from CSV to Parquet

This blueprint builds an end-to-end file processing pipeline in Kestra: it downloads a compressed ZIP archive over HTTP, decompresses it, reads the CSV inside, and writes the records to a schema-validated Parquet file. It solves the common ingestion problem of turning raw, zipped CSV exports into a typed, columnar format that is ready for analytics, data lakes, and downstream warehouse loads, without any glue scripts or manual steps.

How it works

1. get_zipfile (io.kestra.plugin.core.http.Download) fetches the Divvy bikeshare ZIP archive over HTTP. The URL is templated with the file_id variable so you can swap datasets by changing one value.
2. unzip (io.kestra.plugin.compress.ArchiveDecompress) decompresses the archive using the ZIP algorithm and exposes the extracted files as outputs.
3. csv_to_ion (io.kestra.plugin.serdes.csv.CsvToIon) reads the extracted CSV and converts it to Kestra's internal Ion format for efficient, streamable processing.
4. to_parquet (io.kestra.plugin.serdes.avro.IonToAvro) writes the data to Parquet, enforcing an explicit Avro schema with typed fields (strings, doubles, nullable coordinates, and timestamp-millis columns) and a datetimeFormat for parsing dates.

What you get

• A repeatable HTTP-to-Parquet pipeline with no custom code.
• Strong typing and schema validation via an explicit Avro schema.
• Internal storage handoff between tasks through Kestra outputs ({{ outputs.get_zipfile.uri }} and friends).
• A columnar Parquet output ready for warehouse and lakehouse loads.

Who it's for

• Data engineers ingesting third-party CSV exports.
• Analytics teams standardizing raw files into typed Parquet.
• Anyone replacing ad hoc download-and-convert scripts with a declarative flow.

Why orchestrate this with Kestra

Kestra turns a chain of shell steps into a declarative YAML workflow with built-in retries, full execution lineage, and event-driven or scheduled triggers. The file utilities themselves (download, unzip, CSV and Parquet conversion) have no scheduler, no state, and no observability: Kestra supplies the orchestration layer, passing files between tasks through internal storage and giving you replays and per-task logs that standalone serdes tools cannot offer.

Prerequisites

• A running Kestra instance.
• Network access from the worker to the source ZIP URL.

Secrets

This flow uses no secrets. The source file is fetched from a public HTTP endpoint.

Quick start

1. Add the blueprint to your Kestra instance.
2. Execute the flow as-is to download the sample Divvy dataset.
3. Inspect the to_parquet output to confirm the schema-validated Parquet file.

How to extend

• Change the file_id variable to process a different dataset.
• Swap the Avro schema to match your own CSV columns.
• Add a Schedule trigger to run on a cadence, or a webhook trigger for event-driven ingestion.
• Append a warehouse load task (BigQuery, Snowflake, Postgres) to push the Parquet output downstream.

Links

• Core plugin docs: https://kestra.io/plugins/plugin-core
• Compress plugin docs: https://kestra.io/plugins/plugin-compress
• Serdes plugin docs: https://kestra.io/plugins/plugin-serdes