Query Elasticsearch using ES|QL.
This task allows you to query Elasticsearch using ES|QL.
type: "io.kestra.plugin.elasticsearch.Esql"Load data in bulk to Elasticsearch and query it using ES|QL.
id: bulk_load_and_query
namespace: company.team
tasks:
- id: extract
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/resolve/main/jsonl/books.jsonl
- id: load
type: io.kestra.plugin.elasticsearch.Bulk
from: "{{ outputs.extract.uri }}"
- id: sleep
type: io.kestra.plugin.core.flow.Sleep
duration: PT5S
description: Pause needed after load before we can query
- id: query
type: io.kestra.plugin.elasticsearch.Esql
fetchType: STORE
query: |
FROM books
| KEEP author, name, page_count, release_date
| SORT page_count DESC
| LIMIT 5
pluginDefaults:
- type: io.kestra.plugin.elasticsearch
values:
connection:
headers:
- "Authorization: ApiKey yourEncodedApiKey"
hosts:
- https://yourCluster.us-central1.gcp.cloud.es.io:443
The connection properties.
The way you want to store the data.
FETCH_ONE output the first row, FETCH output all the rows, STORE store all rows in a file, NONE do nothing.
The ElasticSearch value.
Can be a JSON string. In this case, the contentType will be used or a raw Map.
Query filter.
Specify a DSL query in the filter parameter to filter the set of documents that an ES|QL query runs on.
Controls the shard routing of the request.
Using this value to hash the shard and not the id.
Map containing the first row of fetched data.
Only populated if using fetchType=FETCH_ONE.
List containing the fetched data.
Only populated if using fetchType=FETCH.
The number of fetched rows.
The total number of rows fetched without pagination.
The URI of the data stored in Kestra's internal storage.
Only populated if using fetchType=STORE.
List of HTTP ElasticSearch servers.
Must be an URI like https://elasticsearch.com: 9200 with scheme and port.
Basic auth configuration.
List of HTTP headers to be send on every request.
Must be a string with key value separated with : , ex: Authorization: Token XYZ.
Sets the path's prefix for every request used by the HTTP client.
For example, if this is set to /my/path, then any client request will become /my/path/ + endpoint.
In essence, every request's endpoint is prefixed by this pathPrefix.
The path prefix is useful for when ElasticSearch is behind a proxy that provides a base path or a proxy that requires all paths to start with '/'; it is not intended for other purposes and it should not be supplied in other scenarios.
Basic auth password.
Basic auth username.