LanceDB Search

LanceDB Search

Certified

Search nearest vectors in a LanceDB table

Searches for nearest neighbor vectors in a LanceDB table with optional SQL metadata filtering. Returns results inline or stores them in Kestra internal storage.

yaml
type: io.kestra.plugin.lancedb.Search

Run similarity search with metadata filtering and return inline matches

yaml
id: lancedb_search
namespace: company.team

tasks:
  - id: search
    type: io.kestra.plugin.lancedb.Search
    uri: "s3://my-bucket/lancedb"
    table: documents
    vector: "{{ outputs.embed.vector }}"
    topK: 5
    filter: "category = 'docs'"
    fetchType: FETCH
    storageOptions:
      aws_access_key_id: "{{ secret('AWS_ACCESS_KEY_ID') }}"
      aws_secret_access_key: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
      region: "us-east-1"

  - id: log_results
    type: io.kestra.plugin.core.log.Log
    message: "Found {{ outputs.search.size }} matches"

Search nearest vectors and store results to internal storage file

yaml
id: lancedb_search_store
namespace: company.team

tasks:
  - id: search
    type: io.kestra.plugin.lancedb.Search
    uri: "/tmp/lancedb"
    table: documents
    vector: [0.05, 0.61, 0.76, 0.74]
    topK: 10
    fetchType: STORE
Properties

Table name

The name of the LanceDB table to search in.

Database URI

Path or URI to the LanceDB database directory or endpoint, e.g. local directory '/tmp/lancedb', cloud object store 's3://my-bucket/lancedb', 'gs://my-bucket/lancedb', 'az://my-bucket/lancedb'.

Query vector

Dense vector embedding to search for nearest neighbors.

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
SubTypestring

Columns to project

Optional list of columns to return in result rows. Default returns all columns.

DefaultFETCH
Possible Values
STOREFETCHFETCH_ONENONE

Fetch type

The way results should be fetched: FETCH_ONE (single map row), FETCH (in-memory list of rows), or STORE (write to Kestra storage URI).

Metadata filter

SQL filter expression for metadata pre-filtering (e.g. category = 'docs' AND rating > 4).

Storage options

Key-value configuration options for cloud object storage (e.g. AWS access key, secret key, region, endpoint URL).

Default10

Top-K results

Maximum number of nearest neighbor results to return, between 1 and 100000 (default: 10).

Map containing the first row of fetched data.

Only populated if using fetchType=FETCH_ONE.

List containing the fetched data.

Only populated if using fetchType=FETCH.

The number of fetched rows.

Formaturi

Kestra's internal storage URI of the stored data.

Only populated if using fetchType=STORE.