Actian Vector (VectorWise) Batch
CertifiedBulk insert rows into Vectorwise using prepared statements
Actian Vector (VectorWise) Batch
Bulk insert rows into Vectorwise using prepared statements
Reads ION-formatted data from Kestra internal storage and performs high-performance batch inserts using JDBC batch operations. Data is processed in chunks (default 1,000 rows) to optimize memory and performance. Optimized for Vectorwise's columnar storage. Supports auto-commit for databases without transaction support.
type: io.kestra.plugin.jdbc.vectorwise.BatchExamples
Fetch rows from a table and bulk insert to another one.
id: vectorwise_batch_query
namespace: company.team
tasks:
- id: query
type: io.kestra.plugin.jdbc.vectorwise.Query
url: jdbc:vectorwise://dev:port/base
username: "{{ secret('VECTORWISE_USERNAME') }}"
password: "{{ secret('VECTORWISE_PASSWORD') }}"
sql: |
SELECT *
FROM xref
LIMIT 1500;
fetchType: STORE
- id: update
type: io.kestra.plugin.jdbc.vectorwise.Batch
from: "{{ outputs.query.uri }}"
url: jdbc:vectorwise://prod:port/base
username: "{{ secret('VECTORWISE_USERNAME') }}"
password: "{{ secret('VECTORWISE_PASSWORD') }}"
sql: insert into xref values( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )
Fetch rows from a table and bulk insert to another one without using sql query.
id: vectorwise_batch_query
namespace: company.team
tasks:
- id: query
type: io.kestra.plugin.jdbc.vectorwise.Query
url: jdbc:vectorwise://dev:port/base
username: admin
password: admin_passwd
sql: |
SELECT *
FROM xref
LIMIT 1500;
fetchType: STORE
- id: update
type: io.kestra.plugin.jdbc.vectorwise.Batch
from: "{{ outputs.query.uri }}"
url: jdbc:vectorwise://prod:port/base
username: admin
password: admin_passwd
table: xref
Properties
from *string
Input file from internal storage
URI of the source file (kestra://) containing rows to insert
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
url *string
The JDBC URL to connect to the database
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150chunk integerstring
1000Batch size per executeBatch call
Number of rows sent per JDBC batch before commit; default 1,000
columns array
Columns bound to placeholders
Ordered column names matching ? placeholders; if omitted, placeholder count must match all columns in the input row
password string
The database user's password.
sql string
Parameterized INSERT statement to execute
Prepared INSERT with ? placeholders for each bound column. Example: INSERT INTO <table_name> VALUES (?, ?, ?) for three columns; use column list if inserting a subset
table string
Table used to auto-discover columns
Retrieves column names from the given table when columns is empty.
If sql is also omitted, an INSERT statement is generated automatically using the discovered columns
timeZoneId string
The time zone id to use for date/time manipulation. Default value is the worker's default time zone id.
username string
The database user
Outputs
rowCount integer
Total rows read
updatedCount integer
Rows inserted or updated
Metrics
query counter
queriesThe number of batch queries executed.
records counter
recordsThe number of records processed.
updated counter
recordsThe number of records updated.
