
Qdrant HybridQuery
CertifiedQuery a Qdrant collection with dense and sparse vectors fused by RRF or DBSF
Qdrant HybridQuery
Query a Qdrant collection with dense and sparse vectors fused by RRF or DBSF
Runs one server-side hybrid query against a collection that holds both a named dense vector space and a named sparse vector space.
The task sends a single QueryPoints request with two prefetch branches, one per vector space, each bounded by its own candidate limit, and asks Qdrant to merge the two candidate lists with native Reciprocal Rank Fusion by default, or Distribution-Based Score Fusion with fusion: DBSF. Fusion happens inside Qdrant, so no scoring is done by Kestra.
The collection must already expose the dense and the sparse vector space under the names given here. Create it with the CreateCollection task (vectorName and sparseVectors) and load it with the Upsert task, which accepts sparse vectors. Hybrid retrieval is opt-in: the existing Query task is unchanged, and fusion does not improve every workload, so compare it with dense-only retrieval on your own labelled queries before adopting it.
type: io.kestra.plugin.qdrant.HybridQueryExamples
Fuse a dense branch and a sparse branch with server-side RRF, filtering both branches and the fused result. The docs collection must already hold points with a text_dense dense vector and a text_sparse sparse vector.
id: qdrant_hybrid_query
namespace: company.team
inputs:
- id: dense_embedding
type: ARRAY
itemType: FLOAT
defaults: [0.9, 0.1, 0.0, 0.0]
tasks:
- id: hybrid_search
type: io.kestra.plugin.qdrant.HybridQuery
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: docs
denseVectorName: text_dense
denseVector: "{{ inputs.dense_embedding }}"
sparseVectorName: text_sparse
sparseIndices: [12, 104, 982]
sparseValues: [0.84, 1.23, 0.41]
candidateLimit: 50
limit: 10
filter:
tenant_id: acme
withPayload: true
fetchType: FETCH
- id: log_hits
type: io.kestra.plugin.core.log.Log
message: "{{ outputs.hybrid_search.rows }}"
Tune the fusion: weight the sparse branch higher in RRF (needs Qdrant 1.17 or later), or switch to DBSF, and read the second page of results
id: qdrant_hybrid_query_tuned
namespace: company.team
tasks:
- id: rrf_sparse_heavy
type: io.kestra.plugin.qdrant.HybridQuery
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: docs
denseVectorName: text_dense
denseVector: [0.9, 0.1, 0.0, 0.0]
sparseVectorName: text_sparse
sparseIndices: [12, 104, 982]
sparseValues: [0.84, 1.23, 0.41]
rrfK: 5
denseWeight: 1.0
sparseWeight: 2.0
limit: 10
fetchType: FETCH
- id: dbsf_second_page
type: io.kestra.plugin.qdrant.HybridQuery
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: docs
denseVectorName: text_dense
denseVector: [0.9, 0.1, 0.0, 0.0]
sparseVectorName: text_sparse
sparseIndices: [12, 104, 982]
sparseValues: [0.84, 1.23, 0.41]
fusion: DBSF
limit: 10
offset: 10
candidateLimit: 50
fetchType: FETCH
Evaluate dense-only, sparse-only and fused retrieval on a labelled query set. The flow creates a throwaway collection, loads a small corpus, scores hit@k per method and always drops the collection.
id: qdrant_retrieval_evaluation
namespace: company.team
variables:
collection: "eval_{{ execution.id | lower }}"
inputs:
- id: k
type: INT
defaults: 3
description: A query counts as a hit when its relevant point is among the first k results of a method.
- id: queries
type: JSON
description: Labelled queries. Each one has a dense vector, a sparse vector and the id of the point that is relevant for it.
defaults: |
[
{"name": "exact identifier", "dense": [0.0, 0.3, 0.7, 0.0], "indices": [13], "values": [1.0], "relevant": 4},
{"name": "paraphrase", "dense": [0.1, 0.9, 0.0, 0.0], "indices": [99], "values": [1.0], "relevant": 3},
{"name": "both signals", "dense": [0.9, 0.1, 0.0, 0.0], "indices": [10], "values": [1.0], "relevant": 1},
{"name": "keyword only", "dense": [0.25, 0.25, 0.25, 0.25], "indices": [14], "values": [1.0], "relevant": 5}
]
tasks:
- id: create_collection
type: io.kestra.plugin.qdrant.CreateCollection
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: "{{ render(vars.collection) }}"
vectorName: text_dense
vectorSize: 4
distance: COSINE
sparseVectors:
- name: text_sparse
- id: load_corpus
type: io.kestra.plugin.qdrant.Upsert
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: "{{ render(vars.collection) }}"
points:
- id: 1
vectors:
text_dense: [1.0, 0.0, 0.0, 0.0]
text_sparse: {indices: [10, 20], values: [1.0, 0.5]}
- id: 2
vectors:
text_dense: [0.9, 0.1, 0.0, 0.0]
text_sparse: {indices: [11], values: [1.0]}
- id: 3
vectors:
text_dense: [0.0, 1.0, 0.0, 0.0]
text_sparse: {indices: [12, 20], values: [1.0, 0.5]}
- id: 4
vectors:
text_dense: [0.0, 0.1, 0.9, 0.0]
text_sparse: {indices: [13], values: [2.0]}
- id: 5
vectors:
text_dense: [0.0, 0.0, 1.0, 0.0]
text_sparse: {indices: [14], values: [1.0]}
- id: 6
vectors:
text_dense: [0.0, 0.0, 0.1, 0.9]
text_sparse: {indices: [15, 13], values: [1.0, 0.2]}
- id: evaluate
type: io.kestra.plugin.core.flow.ForEach
values: "{{ inputs.queries }}"
tasks:
- id: dense_only
type: io.kestra.plugin.qdrant.Query
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: "{{ render(vars.collection) }}"
vectorName: text_dense
vector: "{{ fromJson(taskrun.value).dense }}"
limit: "{{ inputs.k }}"
fetchType: FETCH
- id: sparse_only
type: io.kestra.plugin.qdrant.SparseQuery
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: "{{ render(vars.collection) }}"
vectorName: text_sparse
indices: "{{ fromJson(taskrun.value).indices }}"
values: "{{ fromJson(taskrun.value).values }}"
limit: "{{ inputs.k }}"
fetchType: FETCH
- id: fused
type: io.kestra.plugin.qdrant.HybridQuery
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: "{{ render(vars.collection) }}"
denseVectorName: text_dense
denseVector: "{{ fromJson(taskrun.value).dense }}"
sparseVectorName: text_sparse
sparseIndices: "{{ fromJson(taskrun.value).indices }}"
sparseValues: "{{ fromJson(taskrun.value).values }}"
limit: "{{ inputs.k }}"
candidateLimit: 6
fetchType: FETCH
- id: score
type: io.kestra.plugin.core.output.OutputValues
values:
name: "{{ fromJson(taskrun.value).name }}"
dense_hit: "{{ outputs.dense_only[taskrun.value].rows | jq('if (map(.id) | index(' ~ fromJson(taskrun.value).relevant ~ ')) != null then 1 else 0 end') | first }}"
sparse_hit: "{{ outputs.sparse_only[taskrun.value].rows | jq('if (map(.id) | index(' ~ fromJson(taskrun.value).relevant ~ ')) != null then 1 else 0 end') | first }}"
fused_hit: "{{ outputs.fused[taskrun.value].rows | jq('if (map(.id) | index(' ~ fromJson(taskrun.value).relevant ~ ')) != null then 1 else 0 end') | first }}"
- id: summary
type: io.kestra.plugin.core.output.OutputValues
values:
queries: "{{ outputs.score | jq('length') | first }}"
dense_hits: "{{ outputs.score | jq('[.[].values.dense_hit | tonumber] | add') | first }}"
sparse_hits: "{{ outputs.score | jq('[.[].values.sparse_hit | tonumber] | add') | first }}"
fused_hits: "{{ outputs.score | jq('[.[].values.fused_hit | tonumber] | add') | first }}"
- id: report
type: io.kestra.plugin.core.log.Log
message: |
hit@{{ inputs.k }} on {{ outputs.summary.values.queries }} labelled queries
dense only : {{ outputs.summary.values.dense_hits }}
sparse only: {{ outputs.summary.values.sparse_hits }}
fused (RRF): {{ outputs.summary.values.fused_hits }}
finally:
- id: drop_collection
type: io.kestra.plugin.qdrant.DeleteCollection
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: "{{ render(vars.collection) }}"
Compare dense-only retrieval with fused retrieval on the same query, then store the fused ranking
id: qdrant_hybrid_query_comparison
namespace: company.team
tasks:
- id: dense_only
type: io.kestra.plugin.qdrant.Query
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: docs
vectorName: text_dense
vector: [0.9, 0.1, 0.0, 0.0]
limit: 10
fetchType: FETCH
- id: hybrid
type: io.kestra.plugin.qdrant.HybridQuery
host: "{{ secret('QDRANT_HOST') }}"
apiKey: "{{ secret('QDRANT_API_KEY') }}"
tlsEnabled: true
collectionName: docs
denseVectorName: text_dense
denseVector: [0.9, 0.1, 0.0, 0.0]
sparseVectorName: text_sparse
sparseIndices: [12, 104, 982]
sparseValues: [0.84, 1.23, 0.41]
denseCandidateLimit: 100
sparseCandidateLimit: 100
limit: 10
queryTimeout: PT10S
fetchType: STORE
- id: compare
type: io.kestra.plugin.core.log.Log
message: |
dense-only ids: {{ outputs.dense_only.rows | jq('[.[].id]') | first }}
fused ranking stored at: {{ outputs.hybrid.uri }}
Installation
Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:
kestractl plugins get io.kestra.plugin:plugin-qdrant --compatible-for $KESTRA_VERSION For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.
Properties
collectionName *string
Collection name
The name of the collection to search in. It must expose both the dense and the sparse vector space named below.
denseVector *array
Dense query vector
Dense query embedding, as a list of finite numbers. Its dimension must match the dense vector space of the collection.
denseVectorName *string
Dense vector space name
Name of the dense vector space to use for the dense prefetch branch, as configured on the collection.
host *string
The host of the Qdrant instance
The hostname or IP address of the Qdrant gRPC endpoint.
sparseIndices *array
Sparse query indices
Indices of the non-zero dimensions of the sparse query vector, as a list of distinct non-negative whole numbers. Must have the same number of elements as 'sparseValues'.
sparseValues *array
Sparse query values
Weights of the non-zero dimensions of the sparse query vector, as a list of finite numbers positionally paired with 'sparseIndices'.
sparseVectorName *string
Sparse vector space name
Name of the sparse vector space to use for the sparse prefetch branch, as configured on the collection.
apiKey string
API key
The API key for authentication with Qdrant.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150candidateLimit integerstring
50Candidate limit per branch
Maximum number of candidates each prefetch branch returns to the fusion step (default: 50, min: 1, max: 10000). Must be greater than or equal to 'limit' plus 'offset'. Overridden per branch by 'denseCandidateLimit' and 'sparseCandidateLimit'.
denseCandidateLimit integerstring
Candidate limit of the dense branch
Overrides 'candidateLimit' for the dense prefetch branch only (min: 1, max: 10000). Must be greater than or equal to 'limit'.
denseWeight numberstring
RRF weight of the dense branch
Weight of the dense branch in Reciprocal Rank Fusion (min: 0, max: 1000). Weights are absolute, not relative: 1 and 2 is a different setting from 2 and 4, because the position is divided by the weight. A weight of 0 keeps the branch candidates at the bottom with a score of 0. Unset keeps the Qdrant default of 1. Only valid with fusion: RRF, must be set together with sparseWeight, and needs Qdrant 1.17 or later.
fetchType string
STORESTOREFETCHFETCH_ONENONESelect fetch behavior
Output mode for query results: FETCH returns all rows inline, FETCH_ONE returns the first row, and STORE writes all rows to Kestra storage as an ION file and returns the URI (default: STORE).
filter object
Filter criteria
Filter map applied to both prefetch branches and to the fused result, so a point excluded by the filter cannot enter either candidate list. Accepts the same filter shapes as the 'Query' task.
fusion string
RRFRRFDBSFFusion method
How the two candidate lists are merged: RRF (default) scores a point from its rank in each branch, DBSF normalizes each branch score distribution and sums the normalized scores and needs Qdrant 1.11 or later. The returned score is the fusion score for either method.
limit integerstring
10Limit fused results
Maximum number of points returned after fusion (default: 10, min: 1, max: 1000).
offset integerstring
0Skip fused results
Number of fused results to skip before returning limit points, for pagination (default: 0, max: 10000). Every candidate limit must be at least limit + offset, because fusion has to rank the skipped points too.
port integerstring
6334The port of the Qdrant instance
The gRPC port of the Qdrant instance (default 6334).
queryTimeout string
PT30SQuery timeout
Server-side and client-side bound for the hybrid query, truncated to whole seconds (default: PT30S, min: 1 second). The same value is sent as the Qdrant request timeout and used as the gRPC call deadline.
rrfK integerstring
RRF rank constant
The k constant of Reciprocal Rank Fusion (min: 1, max: 10000). With equal weights a point at zero-based position pos in a branch scores 1 / (pos + k), so a larger k flattens the gap between top ranks. Unset keeps the Qdrant default of 2. Use 61 to reproduce the 1 / (rank + 60) convention of the original RRF paper. Only valid with fusion: RRF, and needs Qdrant 1.17 or later.
sparseCandidateLimit integerstring
Candidate limit of the sparse branch
Overrides 'candidateLimit' for the sparse prefetch branch only (min: 1, max: 10000). Must be greater than or equal to 'limit'.
sparseWeight numberstring
RRF weight of the sparse branch
Weight of the sparse branch in Reciprocal Rank Fusion (min: 0, max: 1000). Weights are absolute, not relative, as described for denseWeight. Unset keeps the Qdrant default of 1. Only valid with fusion: RRF, must be set together with denseWeight, and needs Qdrant 1.17 or later.
tlsEnabled booleanstring
falseEnable TLS
Whether to use TLS/SSL for secure gRPC connection.
withPayload booleanstring
trueInclude payload
Whether to include point payload metadata in the result (default: true).
withVectors booleanstring
falseInclude vectors
Whether to include vector embeddings in the result (default: false).
Outputs
row object
Map containing the first row of fetched data.
Only populated if using fetchType=FETCH_ONE.
rows array
List containing the fetched data.
Only populated if using fetchType=FETCH.
size integer
The number of fetched rows.
uri string
uriKestra's internal storage URI of the stored data.
Only populated if using fetchType=STORE.