🚀 New! Kestra raises $3 million to grow Learn more

Home / Solutions / Blueprints

Extract data from Apache Pinot into a CSV format and load it into BigQuery

Plugins Used in this Blueprints

About this blueprint

Extract data from Apache Pinot into a CSV format and load it into BigQuery

This flow extracts data from an Apache Pinot table, writes it to a CSV file, and then loads the CSV data into BigQuery.

Note that we convert the bytes column location_st_point into a STRING format while extracting the data from Pinot.

The GCP credentials are configured using an environment variable GCP_SERVICE_ACCOUNT_JSON. The BigQuery table will get created at runtime using the schema provided in the flow code.

You can setup Apache Pinot locally with Docker using:

bash
docker pull apachepinot/pinot:latest
docker run -p 9000:9000 apachepinot/pinot:latest QuickStart -type batch

The Quickstart mode will automatically load the example datasets. One of the example datasets starbucksStores is used in this flow.

More Related Blueprints

New to Kestra?

Use blueprints to kickstart your first workflows.