Quickwit Search

Quickwit Search

Certified

Search Quickwit indexes

Runs a Quickwit search query against an index and returns the matching documents.

Supports returning every hit, only the first one, or storing all hits to Kestra internal storage. Only the current result page is returned, so set maxHits (and paginate) when you need more.

yaml
type: io.kestra.plugin.quickwit.search.Search

Search the errors logged in the last hour

yaml
id: quickwit_search
namespace: company.team

tasks:
  - id: search
    type: io.kestra.plugin.quickwit.search.Search
    url: "https://quickwit.example.com:7280"
    index: app-logs
    query: "severity:ERROR"
    startTimestamp: "{{ now() | dateAdd(-1, 'HOURS') | timestamp }}"
    maxHits: 100
    fetchType: FETCH

  - id: log_count
    type: io.kestra.plugin.core.log.Log
    message: "Found {{ outputs.search.size }} errors"

Count the errors of each service over the last day, behind a protected gateway

yaml
id: quickwit_search_aggregations
namespace: company.team

tasks:
  - id: search
    type: io.kestra.plugin.quickwit.search.Search
    url: "https://quickwit.example.com:7280"
    headers:
      Authorization: "Bearer {{ secret('QUICKWIT_GATEWAY_TOKEN') }}"
    index: app-logs
    query: "severity:ERROR"
    startTimestamp: "{{ now() | dateAdd(-1, 'DAYS') | timestamp }}"
    aggregations:
      service:
        terms:
          field: service

  - id: report
    type: io.kestra.plugin.core.log.Log
    message: "Top services: {{ outputs.search.aggregations | json }}"
Properties

Index ID

ID of the index to search.

Quickwit also accepts a multi-target expression such as app-logs-000001,app-logs-000002 or a wildcard pattern such as app-logs*.

Query

Query text, using the Quickwit query language, for example severity: ERROR.

Quickwit URL

Base URL of the Quickwit REST API, including the scheme. The api/v1 prefix is added automatically. A local node listens on port 7280, e.g. http://localhost: 7280.

Aggregations

Aggregation request, using the aggregations syntax, for example {"severity": {"terms": {"field": "severity"}}}.

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150

Basic authentication

Optional HTTP basic authentication credentials.

Quickwit itself has no authentication layer, so this is only needed when the cluster sits behind a reverse proxy or an API gateway that enforces basic auth.

Definitions
passwordstring

Password

Password for HTTP basic authentication.

usernamestring

Username

Username for HTTP basic authentication.

Connect timeout

Maximum time to wait to establish a connection to Quickwit, e.g. PT10S. When unset, the default of the underlying HTTP client applies.

End timestamp

Restricts the search to documents with timestamp < endTimestamp, in seconds. Taking advantage of this lets Quickwit prune splits.

DefaultFETCH
Possible Values
STOREFETCHFETCH_ONENONE

Result handling mode

Controls how hits are exposed in the outputs.

FETCH (default) returns every hit of the current page, FETCH_ONE only the first hit, STORE writes all hits to Kestra internal storage and returns a URI, and NONE leaves the row outputs empty.

Custom HTTP headers

Headers sent on every request, for example an Authorization: Bearer ... token or a gateway API key.

Quickwit itself has no authentication layer, so this is only needed when the cluster sits behind an API gateway that injects credentials.

Maximum number of hits

Maximum number of documents to return. Quickwit defaults to 20.

Read timeout

Maximum time to keep waiting for data on an idle connection, e.g. PT5M. A commit=force ingest or a large search may legitimately take a while, so raise it when needed.

SubTypestring

Search fields

Fields to search on when the query does not qualify a field name. Defaults to the default_search_fields of the index configuration.

SubTypestring

Snippet fields

Fields to extract snippets on. Snippets are returned in the snippets output.

SubTypestring

Sort by

Fields to sort the results on. Prefix a field with - to sort ascending or + to sort descending. You can sort by up to two fast fields, or by the BM25 _score. By default Quickwit returns the most recent documents first.

Start offset

Number of documents to skip. Quickwit defaults to 0.

Start timestamp

Restricts the search to documents with timestamp >= startTimestamp, in seconds. Taking advantage of this lets Quickwit prune splits.

Aggregations

Aggregation results, present only when aggregations were requested.

Processing time

Time taken by Quickwit to process the query, in microseconds.

SubTypestring

Search errors

Non-fatal errors reported by Quickwit while executing the query.

First document

Available only when fetchType=FETCH_ONE; the first matching document.

SubTypeobject

Fetched documents

Available only when fetchType=FETCH; the documents of the current result page.

Returned document count

Number of documents included in the outputs for the selected fetch type.

SubTypeobject

Snippets

Extracted snippets, present only when snippet fields were requested.

Total matching documents

Total number of documents matching the query, regardless of pagination.

Formaturi

Stored documents URI

Available only when fetchType=STORE; path of the ION file holding the documents.

Unitrecords

Number of documents returned

Number of search requests sent

Time taken by Quickwit to process the query