Apache Hudi Delete
Certified
Delete data from an Apache Hudi table
Apache Hudi Delete
Delete data from an Apache Hudi table
Delete records from an Apache Hudi table using HoodieJavaWriteClient. On a partitioned table, from must contain the partition path of each record, otherwise the task fails instead of deleting nothing.
type: io.kestra.plugin.hudi.DeleteExamples
Delete records from an Apache Hudi table
id: hudi_delete_flow
namespace: company.team
tasks:
- id: delete_records
type: io.kestra.plugin.hudi.Delete
basePath: hdfs://namenode:8020/data/hudi/orders
tableName: orders
recordKeyField: order_id
from:
- order_id: 101
Installation
Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:
kestractl plugins get io.kestra.plugin:plugin-hudi --compatible-for $KESTRA_VERSION For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.
Properties
basePath *string
The base path of the Hudi table
Path where the Hudi table is located (e.g., orders, hdfs://namenode: 8020/data/hudi/orders). Allowed remote schemes: hdfs, s3a, gs, abfs, abfss, wasb, wasbs. Local paths (relative, absolute or file://) must be inside the working directory, and relative paths are resolved against it. A local table only lives as long as the working directory of the task. Remote s3a, gs and abfs paths only work when the matching Hadoop connector (hadoop-aws, gcs-connector, hadoop-azure) is added to the worker classpath, it is not bundled.
from *object
Source keys or records
Keys or records to delete. Can be a list of maps, strings, or a Kestra internal storage URI (kestra://...). When partitionPathField is set (partitioned table) every entry must be a record containing both the record key field and the partition path field, because a bare key cannot be located; the task fails otherwise. Bare keys only work on non-partitioned tables.
recordKeyField *string
Record key field
Field name to extract the record key from incoming records.
tableName *string
The name of the Hudi table
Target Apache Hudi table name.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150configuration object
Configuration properties for the Hudi connection
Key-value configuration map (e.g., Hadoop/AWS/GCS credentials, Hudi write configs). Allowed configuration keys start with hoodie., fs.s3a., fs.gs., fs.azure. or fs.abfs.. Within hoodie., only write and table tuning keys are accepted: record key, precombine and partition path fields, table name and type, index type, insert, upsert, bulk insert and delete parallelism, hoodie.cleaner.*, hoodie.clean.*, hoodie.compact.*, hoodie.clustering.*, hoodie.archive.*, hoodie.keep.*, hoodie.parquet.*, hoodie.logfile.*, hoodie.copyonwrite.* and hoodie.index.bloom.*. Other hoodie. keys are rejected, as is any key ending with .path, .dir or .pattern, or containing .class, .impl, .factory or classname. Within the fs.* prefixes, keys ending with .impl, .class, .classname or .factory, and keys containing provider, signers, interceptors, binding, keyfile or keytab, are not allowed; the credential and endpoint keys access.key, secret.key, session.token, endpoint, path.style.access and region (e.g. fs.s3a.access.key) are always accepted. fs.defaultFS and fs.viewfs.* are not allowed, the scheme comes from basePath. Values must be literal: variable expansion is not applied and values containing ${ are rejected.
partitionPathField string
Partition path field
Field name to extract the partition path. Defaults to empty for non-partitioned tables.
Outputs
affectedPartitions array
Affected partition paths
List of partition paths modified during the delete operation.
commitInstantTime string
Commit instant timestamp
The instant timestamp generated for this commit on the Hudi active timeline.
recordCount integer
Number of records deleted
Number of records actually deleted, as reported by the Hudi write statuses. Keys that do not exist are not counted.