Core Plugins and tasks FilterItems
CertifiedFilter line-oriented files with a Pebble expression.
Core Plugins and tasks FilterItems
Filter line-oriented files with a Pebble expression.
Reads a line-delimited file from internal storage, evaluates filterCondition per item, and writes matched lines to a new file. filterType controls include vs exclude; errorOrNullBehavior handles expression errors or nulls.
Expressions can reference columns directly (rendered as strings unless cast) and must return a boolean.
type: io.kestra.plugin.core.storage.FilterItemsExamples
Filter a CSV file and retain rows with a product ID equal to 20.
id: filter_items
namespace: company.team
tasks:
- id: download
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
- id: csv_to_ion
type: io.kestra.plugin.serdes.csv.CsvToIon
from: "{{ outputs.download.uri }}"
- id: filter
type: io.kestra.plugin.core.storage.FilterItems
from: "{{ outputs.download.uri }}"
filterCondition: "{{ (product_id | number) == 20 }}"
filterType: INCLUDE
Properties
filterCondition *string
The expression used to match items to be included or excluded
Headers from the file can be referenced directly, e.g., {{ product_id }}, but will be rendered as a string unless combined with a filter, e.g., product_id | number. The Pebble expression should return a BOOLEAN value (i.e., true or false). Values 0, -0, and "" are interpreted as false. Otherwise, any non empty value will be interpreted as true.
from *string
The file to be filtered
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150errorOrNullBehavior string
FAILFAILINCLUDEEXCLUDESpecifies the behavior when the expression fails to be evaluated on an item or returns null.
Use FAIL to throw the exception and fail the task, INCLUDE to pass the item through, or EXCLUDE to drop the item.
filterType string
INCLUDEINCLUDEEXCLUDESpecifies the action to perform with items that match the filterCondition predicate
Use INCLUDE to pass the item through, or EXCLUDE to drop the items.
Outputs
droppedItemsTotal integer
The total number of items that were dropped by the task
processedItemsTotal integer
The total number of items that were processed by the task
uri string
uriThe filtered file URI
