Quickwit Create
CertifiedCreate a Quickwit index
Quickwit Create
Create a Quickwit index
Creates an index from a doc mapping, and optional indexing settings, search settings and retention policy.
Quickwit also accepts the index configuration as a YAML document. This task sends JSON, which cannot carry YAML comments or anchors.
type: io.kestra.plugin.quickwit.index.CreateExamples
Create a logs index with a one week retention
id: quickwit_create_index
namespace: company.team
tasks:
- id: create
type: io.kestra.plugin.quickwit.index.Create
url: "http://localhost:7280"
index: app-logs
configVersion: "0.8"
docMapping:
timestamp_field: timestamp
field_mappings:
- name: timestamp
type: datetime
input_formats:
- unix_timestamp
fast: true
- name: service
type: text
fast: true
- name: severity
type: text
fast: true
- name: message
type: text
searchSettings:
default_search_fields:
- message
retention:
period: "7 days"
schedule: "@daily"
- id: created
type: io.kestra.plugin.core.log.Log
message: "Created index {{ outputs.create.indexUid }}"
Properties
docMapping *object
Doc mapping
Doc mapping of the index: how documents are parsed and which fields are indexed and fast. See the doc mapping documentation.
index *string
Index ID
ID of the index. See the index ID validation rules.
url *string
Quickwit URL
Base URL of the Quickwit REST API, including the scheme. The api/v1 prefix is added automatically.
A local node listens on port 7280, e.g. http://localhost: 7280.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150basicAuth
Basic authentication
Optional HTTP basic authentication credentials.
Quickwit itself has no authentication layer, so this is only needed when the cluster sits behind a reverse proxy or an API gateway that enforces basic auth.
io.kestra.plugin.quickwit.AbstractQuickwitTask-BasicAuth
Password
Password for HTTP basic authentication.
Username
Username for HTTP basic authentication.
configVersion string
Configuration format version
Version of the index configuration format, which must match the version of your Quickwit
cluster, for example 0.8.
Named configVersion and not version because version is reserved by Kestra to pin a
plugin version.
connectTimeout string
Connect timeout
Maximum time to wait to establish a connection to Quickwit, e.g. PT10S.
When unset, the default of the underlying HTTP client applies.
headers object
Custom HTTP headers
Headers sent on every request, for example an Authorization: Bearer ... token or a gateway API key.
Quickwit itself has no authentication layer, so this is only needed when the cluster sits behind an API gateway that injects credentials.
indexUri string
Index URI
Storage URI holding the index files, for example s3://my-bucket/logs.
Defaults to {default_index_root_uri}/{index} on the Quickwit side.
indexingSettings object
Indexing settings
Merge policy and commit settings of the index. See indexing settings.
readTimeout string
Read timeout
Maximum time to keep waiting for data on an idle connection, e.g. PT5M.
A commit=force ingest or a large search may legitimately take a while, so raise it when needed.
retention object
Retention
Retention policy of the index, for example {period: "30 days", schedule: "@daily"}.
See retention policy.
searchSettings object
Search settings
Search settings of the index, for example default_search_fields.
See search settings.
Outputs
createTimestamp integer
Creation timestamp
Creation timestamp of the index, in seconds.
index string
Index ID
ID of the created index.
indexUid string
Index UID
Server-generated unique identifier of the created index.
metadata
Full metadata of the created index, including its effective configuration and sources.
Index metadata
Configuration and state of a Quickwit index.
Source checkpoints
Last committed checkpoint of each source attached to the index, keyed by source ID.
Creation timestamp
Creation timestamp of the index, in seconds.
Index configuration
The effective index configuration, including its doc mapping, indexing settings, search settings and retention policy.
Index UID
Server-generated unique identifier of the index.
Sources
Sources attached to this index.
Index source
Configuration of a source feeding a Quickwit index.
Number of pipelines
Number of indexing pipelines running per node for this source. Defaults to 1.
Source parameters
Parameters of the source, whose shape depends on the source type.
Source ID
Unique identifier of the source within its index.
Source type
Type of the source: kafka, kinesis or pulsar.
Transform
VRL transformation applied to incoming documents, if any.
Configuration format version
Format version of the source configuration, matching the Quickwit version of the cluster.
Configuration format version
Format version of the index configuration, matching the Quickwit version of the cluster.
