Qdrant HybridQuery

Qdrant HybridQuery

Certified

Query a Qdrant collection with dense and sparse vectors fused by RRF or DBSF

Runs one server-side hybrid query against a collection that holds both a named dense vector space and a named sparse vector space.

The task sends a single QueryPoints request with two prefetch branches, one per vector space, each bounded by its own candidate limit, and asks Qdrant to merge the two candidate lists with native Reciprocal Rank Fusion by default, or Distribution-Based Score Fusion with fusion: DBSF. Fusion happens inside Qdrant, so no scoring is done by Kestra.

The collection must already expose the dense and the sparse vector space under the names given here. Create it with the CreateCollection task (vectorName and sparseVectors) and load it with the Upsert task, which accepts sparse vectors. Hybrid retrieval is opt-in: the existing Query task is unchanged, and fusion does not improve every workload, so compare it with dense-only retrieval on your own labelled queries before adopting it.

yaml
type: io.kestra.plugin.qdrant.HybridQuery

Fuse a dense branch and a sparse branch with server-side RRF, filtering both branches and the fused result. The docs collection must already hold points with a text_dense dense vector and a text_sparse sparse vector.

yaml
id: qdrant_hybrid_query
namespace: company.team

inputs:
  - id: dense_embedding
    type: ARRAY
    itemType: FLOAT
    defaults: [0.9, 0.1, 0.0, 0.0]

tasks:
  - id: hybrid_search
    type: io.kestra.plugin.qdrant.HybridQuery
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: docs
    denseVectorName: text_dense
    denseVector: "{{ inputs.dense_embedding }}"
    sparseVectorName: text_sparse
    sparseIndices: [12, 104, 982]
    sparseValues: [0.84, 1.23, 0.41]
    candidateLimit: 50
    limit: 10
    filter:
      tenant_id: acme
    withPayload: true
    fetchType: FETCH

  - id: log_hits
    type: io.kestra.plugin.core.log.Log
    message: "{{ outputs.hybrid_search.rows }}"

Tune the fusion: weight the sparse branch higher in RRF (needs Qdrant 1.17 or later), or switch to DBSF, and read the second page of results

yaml
id: qdrant_hybrid_query_tuned
namespace: company.team

tasks:
  - id: rrf_sparse_heavy
    type: io.kestra.plugin.qdrant.HybridQuery
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: docs
    denseVectorName: text_dense
    denseVector: [0.9, 0.1, 0.0, 0.0]
    sparseVectorName: text_sparse
    sparseIndices: [12, 104, 982]
    sparseValues: [0.84, 1.23, 0.41]
    rrfK: 5
    denseWeight: 1.0
    sparseWeight: 2.0
    limit: 10
    fetchType: FETCH

  - id: dbsf_second_page
    type: io.kestra.plugin.qdrant.HybridQuery
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: docs
    denseVectorName: text_dense
    denseVector: [0.9, 0.1, 0.0, 0.0]
    sparseVectorName: text_sparse
    sparseIndices: [12, 104, 982]
    sparseValues: [0.84, 1.23, 0.41]
    fusion: DBSF
    limit: 10
    offset: 10
    candidateLimit: 50
    fetchType: FETCH

Evaluate dense-only, sparse-only and fused retrieval on a labelled query set. The flow creates a throwaway collection, loads a small corpus, scores hit@k per method and always drops the collection.

yaml
id: qdrant_retrieval_evaluation
namespace: company.team

variables:
  collection: "eval_{{ execution.id | lower }}"

inputs:
  - id: k
    type: INT
    defaults: 3
    description: A query counts as a hit when its relevant point is among the first k results of a method.
  - id: queries
    type: JSON
    description: Labelled queries. Each one has a dense vector, a sparse vector and the id of the point that is relevant for it.
    defaults: |
      [
        {"name": "exact identifier", "dense": [0.0, 0.3, 0.7, 0.0], "indices": [13], "values": [1.0], "relevant": 4},
        {"name": "paraphrase", "dense": [0.1, 0.9, 0.0, 0.0], "indices": [99], "values": [1.0], "relevant": 3},
        {"name": "both signals", "dense": [0.9, 0.1, 0.0, 0.0], "indices": [10], "values": [1.0], "relevant": 1},
        {"name": "keyword only", "dense": [0.25, 0.25, 0.25, 0.25], "indices": [14], "values": [1.0], "relevant": 5}
      ]

tasks:
  - id: create_collection
    type: io.kestra.plugin.qdrant.CreateCollection
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: "{{ render(vars.collection) }}"
    vectorName: text_dense
    vectorSize: 4
    distance: COSINE
    sparseVectors:
      - name: text_sparse

  - id: load_corpus
    type: io.kestra.plugin.qdrant.Upsert
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: "{{ render(vars.collection) }}"
    points:
      - id: 1
        vectors:
          text_dense: [1.0, 0.0, 0.0, 0.0]
          text_sparse: {indices: [10, 20], values: [1.0, 0.5]}
      - id: 2
        vectors:
          text_dense: [0.9, 0.1, 0.0, 0.0]
          text_sparse: {indices: [11], values: [1.0]}
      - id: 3
        vectors:
          text_dense: [0.0, 1.0, 0.0, 0.0]
          text_sparse: {indices: [12, 20], values: [1.0, 0.5]}
      - id: 4
        vectors:
          text_dense: [0.0, 0.1, 0.9, 0.0]
          text_sparse: {indices: [13], values: [2.0]}
      - id: 5
        vectors:
          text_dense: [0.0, 0.0, 1.0, 0.0]
          text_sparse: {indices: [14], values: [1.0]}
      - id: 6
        vectors:
          text_dense: [0.0, 0.0, 0.1, 0.9]
          text_sparse: {indices: [15, 13], values: [1.0, 0.2]}

  - id: evaluate
    type: io.kestra.plugin.core.flow.ForEach
    values: "{{ inputs.queries }}"
    tasks:
      - id: dense_only
        type: io.kestra.plugin.qdrant.Query
        host: "{{ secret('QDRANT_HOST') }}"
        apiKey: "{{ secret('QDRANT_API_KEY') }}"
        tlsEnabled: true
        collectionName: "{{ render(vars.collection) }}"
        vectorName: text_dense
        vector: "{{ fromJson(taskrun.value).dense }}"
        limit: "{{ inputs.k }}"
        fetchType: FETCH

      - id: sparse_only
        type: io.kestra.plugin.qdrant.SparseQuery
        host: "{{ secret('QDRANT_HOST') }}"
        apiKey: "{{ secret('QDRANT_API_KEY') }}"
        tlsEnabled: true
        collectionName: "{{ render(vars.collection) }}"
        vectorName: text_sparse
        indices: "{{ fromJson(taskrun.value).indices }}"
        values: "{{ fromJson(taskrun.value).values }}"
        limit: "{{ inputs.k }}"
        fetchType: FETCH

      - id: fused
        type: io.kestra.plugin.qdrant.HybridQuery
        host: "{{ secret('QDRANT_HOST') }}"
        apiKey: "{{ secret('QDRANT_API_KEY') }}"
        tlsEnabled: true
        collectionName: "{{ render(vars.collection) }}"
        denseVectorName: text_dense
        denseVector: "{{ fromJson(taskrun.value).dense }}"
        sparseVectorName: text_sparse
        sparseIndices: "{{ fromJson(taskrun.value).indices }}"
        sparseValues: "{{ fromJson(taskrun.value).values }}"
        limit: "{{ inputs.k }}"
        candidateLimit: 6
        fetchType: FETCH

      - id: score
        type: io.kestra.plugin.core.output.OutputValues
        values:
          name: "{{ fromJson(taskrun.value).name }}"
          dense_hit: "{{ outputs.dense_only[taskrun.value].rows | jq('if (map(.id) | index(' ~ fromJson(taskrun.value).relevant ~ ')) != null then 1 else 0 end') | first }}"
          sparse_hit: "{{ outputs.sparse_only[taskrun.value].rows | jq('if (map(.id) | index(' ~ fromJson(taskrun.value).relevant ~ ')) != null then 1 else 0 end') | first }}"
          fused_hit: "{{ outputs.fused[taskrun.value].rows | jq('if (map(.id) | index(' ~ fromJson(taskrun.value).relevant ~ ')) != null then 1 else 0 end') | first }}"

  - id: summary
    type: io.kestra.plugin.core.output.OutputValues
    values:
      queries: "{{ outputs.score | jq('length') | first }}"
      dense_hits: "{{ outputs.score | jq('[.[].values.dense_hit | tonumber] | add') | first }}"
      sparse_hits: "{{ outputs.score | jq('[.[].values.sparse_hit | tonumber] | add') | first }}"
      fused_hits: "{{ outputs.score | jq('[.[].values.fused_hit | tonumber] | add') | first }}"

  - id: report
    type: io.kestra.plugin.core.log.Log
    message: |
      hit@{{ inputs.k }} on {{ outputs.summary.values.queries }} labelled queries
      dense only : {{ outputs.summary.values.dense_hits }}
      sparse only: {{ outputs.summary.values.sparse_hits }}
      fused (RRF): {{ outputs.summary.values.fused_hits }}

finally:
  - id: drop_collection
    type: io.kestra.plugin.qdrant.DeleteCollection
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: "{{ render(vars.collection) }}"

Compare dense-only retrieval with fused retrieval on the same query, then store the fused ranking

yaml
id: qdrant_hybrid_query_comparison
namespace: company.team

tasks:
  - id: dense_only
    type: io.kestra.plugin.qdrant.Query
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: docs
    vectorName: text_dense
    vector: [0.9, 0.1, 0.0, 0.0]
    limit: 10
    fetchType: FETCH

  - id: hybrid
    type: io.kestra.plugin.qdrant.HybridQuery
    host: "{{ secret('QDRANT_HOST') }}"
    apiKey: "{{ secret('QDRANT_API_KEY') }}"
    tlsEnabled: true
    collectionName: docs
    denseVectorName: text_dense
    denseVector: [0.9, 0.1, 0.0, 0.0]
    sparseVectorName: text_sparse
    sparseIndices: [12, 104, 982]
    sparseValues: [0.84, 1.23, 0.41]
    denseCandidateLimit: 100
    sparseCandidateLimit: 100
    limit: 10
    queryTimeout: PT10S
    fetchType: STORE

  - id: compare
    type: io.kestra.plugin.core.log.Log
    message: |
      dense-only ids: {{ outputs.dense_only.rows | jq('[.[].id]') | first }}
      fused ranking stored at: {{ outputs.hybrid.uri }}
Installation

Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:

kestractl plugins get io.kestra.plugin:plugin-qdrant --compatible-for $KESTRA_VERSION

For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.

Properties

Collection name

The name of the collection to search in. It must expose both the dense and the sparse vector space named below.

Dense query vector

Dense query embedding, as a list of finite numbers. Its dimension must match the dense vector space of the collection.

Dense vector space name

Name of the dense vector space to use for the dense prefetch branch, as configured on the collection.

The host of the Qdrant instance

The hostname or IP address of the Qdrant gRPC endpoint.

Sparse query indices

Indices of the non-zero dimensions of the sparse query vector, as a list of distinct non-negative whole numbers. Must have the same number of elements as 'sparseValues'.

Sparse query values

Weights of the non-zero dimensions of the sparse query vector, as a list of finite numbers positionally paired with 'sparseIndices'.

Sparse vector space name

Name of the sparse vector space to use for the sparse prefetch branch, as configured on the collection.

API key

The API key for authentication with Qdrant.

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
Default50

Candidate limit per branch

Maximum number of candidates each prefetch branch returns to the fusion step (default: 50, min: 1, max: 10000). Must be greater than or equal to 'limit' plus 'offset'. Overridden per branch by 'denseCandidateLimit' and 'sparseCandidateLimit'.

Candidate limit of the dense branch

Overrides 'candidateLimit' for the dense prefetch branch only (min: 1, max: 10000). Must be greater than or equal to 'limit'.

RRF weight of the dense branch

Weight of the dense branch in Reciprocal Rank Fusion (min: 0, max: 1000). Weights are absolute, not relative: 1 and 2 is a different setting from 2 and 4, because the position is divided by the weight. A weight of 0 keeps the branch candidates at the bottom with a score of 0. Unset keeps the Qdrant default of 1. Only valid with fusion: RRF, must be set together with sparseWeight, and needs Qdrant 1.17 or later.

DefaultSTORE
Possible Values
STOREFETCHFETCH_ONENONE

Select fetch behavior

Output mode for query results: FETCH returns all rows inline, FETCH_ONE returns the first row, and STORE writes all rows to Kestra storage as an ION file and returns the URI (default: STORE).

Filter criteria

Filter map applied to both prefetch branches and to the fused result, so a point excluded by the filter cannot enter either candidate list. Accepts the same filter shapes as the 'Query' task.

DefaultRRF
Possible Values
RRFDBSF

Fusion method

How the two candidate lists are merged: RRF (default) scores a point from its rank in each branch, DBSF normalizes each branch score distribution and sums the normalized scores and needs Qdrant 1.11 or later. The returned score is the fusion score for either method.

Default10

Limit fused results

Maximum number of points returned after fusion (default: 10, min: 1, max: 1000).

Default0

Skip fused results

Number of fused results to skip before returning limit points, for pagination (default: 0, max: 10000). Every candidate limit must be at least limit + offset, because fusion has to rank the skipped points too.

Default6334

The port of the Qdrant instance

The gRPC port of the Qdrant instance (default 6334).

DefaultPT30S

Query timeout

Server-side and client-side bound for the hybrid query, truncated to whole seconds (default: PT30S, min: 1 second). The same value is sent as the Qdrant request timeout and used as the gRPC call deadline.

RRF rank constant

The k constant of Reciprocal Rank Fusion (min: 1, max: 10000). With equal weights a point at zero-based position pos in a branch scores 1 / (pos + k), so a larger k flattens the gap between top ranks. Unset keeps the Qdrant default of 2. Use 61 to reproduce the 1 / (rank + 60) convention of the original RRF paper. Only valid with fusion: RRF, and needs Qdrant 1.17 or later.

Candidate limit of the sparse branch

Overrides 'candidateLimit' for the sparse prefetch branch only (min: 1, max: 10000). Must be greater than or equal to 'limit'.

RRF weight of the sparse branch

Weight of the sparse branch in Reciprocal Rank Fusion (min: 0, max: 1000). Weights are absolute, not relative, as described for denseWeight. Unset keeps the Qdrant default of 1. Only valid with fusion: RRF, must be set together with denseWeight, and needs Qdrant 1.17 or later.

Defaultfalse

Enable TLS

Whether to use TLS/SSL for secure gRPC connection.

Defaulttrue

Include payload

Whether to include point payload metadata in the result (default: true).

Defaultfalse

Include vectors

Whether to include vector embeddings in the result (default: false).

Map containing the first row of fetched data.

Only populated if using fetchType=FETCH_ONE.

List containing the fetched data.

Only populated if using fetchType=FETCH.

The number of fetched rows.

Formaturi

Kestra's internal storage URI of the stored data.

Only populated if using fetchType=STORE.