Analyze salaries of data professionals using a SQL query in DuckDB, extract the results to a CSV file

About this blueprint

SQL Outputs DuckDB

Download a CSV file and query data with DuckDB. Then, extract the results to a CSV file.

yaml

id: duckdb-query
namespace: company.team
tasks:
  - id: download_csv
    type: io.kestra.plugin.core.http.Download
    description: salaries of data professionals from 2020 to 2023 (source ai-jobs.net)
    uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/salaries.csv
  - id: avg_salary_by_job_title
    type: io.kestra.plugin.jdbc.duckdb.Query
    inputFiles:
      data.csv: "{{ outputs.download_csv.uri }}"
    sql: |
      SELECT 
        job_title,
        ROUND(AVG(salary),2) AS avg_salary
      FROM read_csv_auto('{{ workingDir }}/data.csv', header=True)
      GROUP BY job_title
      HAVING COUNT(job_title) > 10
      ORDER BY avg_salary DESC;
    store: true
  - id: result
    type: io.kestra.plugin.serdes.csv.IonToCsv
    from: "{{ outputs.avg_salary_by_job_title.uri }}"

Download

Query

Ion To Csv

More Related Blueprints

Python SQL Outputs API DuckDB

Extract data from a REST API, process it in Python with Polars in a Docker container, then run DuckDB query and preview results as a table in the Outp...

SQL Schedule Outputs Notifications DuckDB

Query a CSV file with DuckDB and send results via Slack every Monday

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra