BulkInsert

type: "io.kestra.plugin.jdbc.clickhouse.BulkInsert"

Bulk Insert new rows into a ClickHouse database.

Examples

Insert rows from another table to a Clickhouse database using asynchronous inserts.

id: clickhouse_bulk_insert
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: bulk_insert
    type: io.kestra.plugin.jdbc.clickhouse.BulkInsert
    from: "{{ inputs.file }}"
    url: jdbc:clickhouse://127.0.0.1:56982/
    username: ch_user
    password: ch_password
    sql: INSERT INTO YourTable SETTINGS async_insert=1, wait_for_async_insert=1 values( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )

Insert data into specific columns via a SQL query to a ClickHouse database using asynchronous inserts.

id: clickhouse_bulk_insert
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: bulk_insert
    type: io.kestra.plugin.jdbc.clickhouse.BulkInsert
    from: "{{ inputs.file }}"
    url: jdbc:clickhouse://127.0.0.1:56982/
    username: ch_user
    password: ch_password
    sql: INSERT INTO YourTable ( field1, field2, field3 ) SETTINGS async_insert=1, wait_for_async_insert=1 values( ?, ?, ? )

Insert data into specific columns via a SQL query to a ClickHouse database using asynchronous inserts.

id: clickhouse_bulk_insert
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: bulk_insert
    type: io.kestra.plugin.jdbc.clickhouse.BulkInsert
    from: "{{ inputs.file }}"
    url: jdbc:clickhouse://127.0.0.1:56982/
    username: ch_user
    password: ch_password
    table: YourTable

Properties

chunk

• Type: integer
• Dynamic: ✔️
• Required: ✔️
• Default: 1000

The size of chunk for every bulk request.

from

• Type: string
• Dynamic: ✔️
• Required: ✔️

Source file URI

sql

• Type: string
• Dynamic: ✔️
• Required: ✔️

Insert query to be executed.

The query must have as many question marks as the number of columns in the table. Example: 'insert into <table_name> values( ? , ? , ? )' for 3 columns. In case you do not want all columns, you need to specify it in the query in the columns property Example: 'insert into <table_name> (id, name) values( ? , ? )' for inserting data into 2 columns: 'id' and 'name'.

url

• Type: string
• Dynamic: ✔️
• Required: ✔️

The JDBC URL to connect to the database.

columns

• Type: array
• SubType: string
• Dynamic: ✔️
• Required: ❌

The columns to be inserted.

If not provided, ? count need to match the from number of columns.

password

• Type: string
• Dynamic: ✔️
• Required: ❌

The database user's password.

table

• Type: string
• Dynamic: ✔️
• Required: ❌

The table from which column names will be retrieved.

This property specifies the table name which will be used to retrieve the columns for the inserted values. You can use it instead of specifying manually the columns in the columns property. In this case, the sql property can also be omitted, an INSERT statement would be generated automatically.

timeZoneId

• Type: string
• Dynamic: ❌
• Required: ❌

The time zone id to use for date/time manipulation. Default value is the worker's default time zone id.

username

• Type: string
• Dynamic: ✔️
• Required: ❌

The database user.

Outputs

rowCount

• Type: integer
• Required: ❌

The rows count.

updatedCount

• Type: integer
• Required: ❌

The updated rows count.