Source
yaml
id: postgres-copyin
namespace: company.team
tasks:
- id: download_products
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/products.csv
- id: create_products_table
type: io.kestra.plugin.jdbc.postgresql.Query
url: "jdbc:postgresql://{{ secret('POSTGRES_HOST') }}:5432/postgres"
username: "{{ secret('POSTGRES_USERNAME') }}"
password: "{{ secret('POSTGRES_PASSWORD') }}"
sql: |
CREATE TABLE IF NOT EXISTS products(
product_id varchar(5),
product_name varchar(100),
product_category varchar(50),
brand varchar(50)
);
- id: copyin_products
type: io.kestra.plugin.jdbc.postgresql.CopyIn
url: "jdbc:postgresql://{{ secret('POSTGRES_HOST') }}:5432/postgres"
username: "{{ secret('POSTGRES_USERNAME') }}"
password: "{{ secret('POSTGRES_PASSWORD') }}"
format: CSV
from: "{{ outputs.download_products.uri }}"
table: products
header: true
delimiter: ","
About this blueprint
Postgres
This flow demonstrates using Postgres CopyIn task to ingest data from CSV file into Postgres table.
The flow has the following tasks: 1. download_products: This task downloads the CSV file containing products data using HTTP download. 2. create_products_table: This task creates the products
table in Postgres if it does not exists. 3. copyin_products: This task copies the data from CSV file dowloaded in the first task into the Postgres table using the Postgres CopyIn task.
More Related Blueprints