Quickwit Trigger
CertifiedTrigger on new Quickwit documents
Quickwit Trigger
Trigger on new Quickwit documents
Polls a Quickwit index on a fixed interval and fires an execution as soon as documents matching the query appear.
The last delivered timestamp is kept in the namespace KV Store and advanced to one past the maximum delivered timestamp on every successful poll, so a given document is only ever delivered once.
type: io.kestra.plugin.quickwit.search.TriggerExamples
Alert when new errors are logged
id: quickwit_error_watch
namespace: company.team
triggers:
- id: on_errors
type: io.kestra.plugin.quickwit.search.Trigger
url: "https://quickwit.example.com:7280"
index: app-logs
query: "severity:ERROR"
timestampField: timestamp
interval: PT5M
tasks:
- id: notify
type: io.kestra.plugin.core.log.Log
message: "New errors: {{ trigger.documents | length }}"
React to new errors on a Quickwit cluster protected by a gateway
id: quickwit_error_watch_private
namespace: company.team
triggers:
- id: on_errors
type: io.kestra.plugin.quickwit.search.Trigger
url: "https://quickwit.example.com:7280"
headers:
Authorization: "Bearer {{ secret('QUICKWIT_GATEWAY_TOKEN') }}"
index: app-logs
query: "severity:ERROR AND service:checkout"
timestampField: timestamp
interval: PT5M
maxHits: 500
tasks:
- id: notify
type: io.kestra.plugin.core.log.Log
message: "New checkout errors: {{ trigger.documents | length }}"
Properties
timestampField *string
Timestamp field
Document field holding the event timestamp, used to advance the watermark.
This should be the timestamp field of the index so start_timestamp pruning aligns.
Its output_format must be rfc3339 (the default) or unix_timestamp_secs, the two formats the watermark can parse.
Polls sort ascending on this field and advance to one past the maximum delivered value.
allowConcurrent boolean
falseSpecifies whether a trigger is allowed to start a new execution even if a previous run is still in progress.
basicAuth
Basic authentication
Optional HTTP basic authentication credentials.
Quickwit itself has no authentication layer, so this is only needed when the cluster sits behind a reverse proxy or an API gateway that enforces basic auth.
io.kestra.plugin.quickwit.AbstractQuickwitTask-BasicAuth
Password
Password for HTTP basic authentication.
Username
Username for HTTP basic authentication.
connectTimeout string
Connect timeout
Maximum time to wait to establish a connection to Quickwit, e.g. PT10S.
When unset, the default of the underlying HTTP client applies.
headers object
Custom HTTP headers
Headers sent on every poll, for example an Authorization: Bearer ... token or a gateway API key.
Quickwit itself has no authentication layer, so this is only needed when the cluster sits behind an API gateway that injects credentials.
index string
Index ID
ID of the index to watch.
interval string
PT1MdurationInterval between polling.
The interval between 2 different polls of schedule, this can avoid to overload the remote system with too many calls. For most of the triggers that depend on external systems, a minimal interval must be at least PT30S. See ISO_8601 Durations for more information of available interval values.
maxHits integerstring
Maximum number of hits
Maximum number of documents to fetch per poll. Quickwit defaults to 20.
Raise it when documents are ingested in bursts. Polls sort ascending on timestampField,
so documents beyond this limit are delivered on a later poll instead of being skipped,
unless a single second holds more documents than this limit.
query string
Query
Query selecting the documents to react to, using the
Quickwit query language, for example severity: ERROR.
readTimeout string
Read timeout
Maximum time to keep waiting for data on an idle connection, e.g. PT5M.
Raise it when the index is large enough for a poll to take a while.
stateKey string
State key
KV Store key holding the watermark of this trigger.
Defaults to <namespace>, <flowId> and <triggerId>, each prefixed by its length, e.g. 7-company_7-my_flow_2-on.
Change it only to deliberately share a watermark between triggers.
stopAfter array
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTEDList of execution states after which a trigger should be stopped (a.k.a. disabled).
url string
Quickwit URL
Base URL of the Quickwit REST API, including the scheme. The api/v1 prefix is added automatically.
A local node listens on port 7280, e.g. http://localhost: 7280.
when string
trueA condition that determines whether the trigger should run.
A Pebble expression evaluated at trigger time. The trigger fires only when the expression evaluates to a truthy value (true, a non-empty string, a non-zero number). Use this to gate trigger execution on dynamic runtime values such as execution labels, flow variables, or environment conditions.
Outputs
documents array
New documents
Documents matching the query that no previous poll had delivered.
elapsedTimeMicros integer
Processing time
Time taken by Quickwit to process the query, in microseconds.
numHits integer
Total matching documents
Total number of documents matching the query over the searched window.
watermark integer
Watermark
Timestamp up to which documents have now been delivered, in seconds.
