Elasticsearch Esql

Elasticsearch Esql

Certified

Run ES|QL query

Executes an ES|QL query and returns results in different formats. Supports synchronous and async execution, optional query parameters (client-side interpolation), and columnar output. Defaults to fetchType=FETCH; STORE writes results to Kestra internal storage. Only the current result set is processed — no pagination.

yaml
type: io.kestra.plugin.elasticsearch.Esql

Aggregate error counts by service using dynamic parameters and async execution

yaml
id: esql_error_report
namespace: company.team

inputs:
  - id: service
    type: STRING
    defaults: "api"
  - id: since
    type: STRING
    defaults: "2024-01-01T00:00:00Z"

tasks:
  - id: query
    type: io.kestra.plugin.elasticsearch.Esql
    fetchType: STORE
    async: true
    connection:
      headers:
        - "Authorization: ApiKey yourEncodedApiKey"
      hosts:
        - https://yourCluster.us-central1.gcp.cloud.es.io:443
    query: |
      FROM logs
      | WHERE service == ? AND @timestamp >= ?
      | STATS error_count = COUNT(*) BY level, host
      | SORT error_count DESC
    params:
      - "{{ inputs.service }}"
      - "{{ inputs.since }}"

Load data in bulk to Elasticsearch and query it using ES|QL

yaml
id: bulk_load_and_query
namespace: company.team

tasks:
  - id: extract
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/resolve/main/jsonl/books.jsonl

  - id: load
    type: io.kestra.plugin.elasticsearch.Bulk
    from: "{{ outputs.extract.uri }}"
    connection:
      headers:
        - "Authorization: ApiKey yourEncodedApiKey"
      hosts:
        - https://yourCluster.us-central1.gcp.cloud.es.io:443

  - id: sleep
    type: io.kestra.plugin.core.flow.Sleep
    duration: PT5S
    description: Pause needed after load before we can query

  - id: query
    type: io.kestra.plugin.elasticsearch.Esql
    fetchType: STORE
    connection:
      headers:
        - "Authorization: ApiKey yourEncodedApiKey"
      hosts:
        - https://yourCluster.us-central1.gcp.cloud.es.io:443
    query: |
      FROM books
        | KEEP author, name, page_count, release_date
        | SORT page_count DESC
        | LIMIT 5
Properties

Elasticsearch connection

Connection settings shared by tasks; hosts are required.

Definitions
hosts*array
SubTypestring
Min items1

Elasticsearch hosts

List of HTTP(S) endpoints including scheme and port, e.g. https://elasticsearch.com: 9200; at least one is required.

basicAuth

Basic authentication

Optional HTTP basic auth credentials rendered at runtime.

passwordstring

Basic auth password

Password for HTTP basic authentication.

usernamestring

Basic auth username

Username for HTTP basic authentication.

connectTimeoutstring

Connection timeout

Maximum time to wait to establish the TCP connection to an Elasticsearch host. When unset, the Elasticsearch client default applies (1 second). A value of PT0S means no timeout.

headersarray
SubTypestring

Custom HTTP headers

Headers sent on every request in Name: Value format, e.g. Authorization: Token XYZ.

pathPrefixstring

Request path prefix

Base path prepended to every Elasticsearch endpoint, e.g. /my/path. Use only when the cluster is served behind a proxy that requires a prefix; leave empty otherwise.

responseTimeoutstring

Response timeout

Maximum time to wait for a response once the request has been sent. When unset, the Elasticsearch client default applies (unbounded), so large bulk requests are not cut off. A value of PT0S means no timeout. Set a bound (e.g. PT60S) to protect against a hung cluster.

strictDeprecationModebooleanstring

Fail on warning headers

When true, any response containing Elasticsearch warning headers is treated as an error.

targetServerVersionintegerstring
Default8

Target Elasticsearch server major version

Major version used for compatibility headers (Accept and Content-Type). Set to 8 for Elasticsearch 8 clusters or 9 for Elasticsearch 9 clusters.

trustAllSslbooleanstring

Trust all SSL certificates

WARNING — SECURITY RISK: When enabled, disables BOTH TLS certificate validation (TrustAllStrategy) AND hostname verification (NoopHostnameVerifier). This makes the connection vulnerable to man-in-the-middle attacks and must never be used in production. Prefer configuring a proper truststore for self-signed certificates instead.

ES|QL query string

ES|QL statement rendered at runtime; required.

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
Defaultfalse

Run query asynchronously

When true, submits the query via the ES|QL async endpoint and polls until complete. Use for long-running analytical queries that would time out on the synchronous endpoint. The query is kept alive for up to 5 minutes. Default is false.

Defaultfalse

Return results in columnar format

When true, uses the ES|QL columnar response format. The rows output then contains one entry per column (not per document): each entry is a single-key map of {columnName: [value1, value2, ...]} where the array holds all values for that column across every matched document. Default is false, which returns one entry per row.

DefaultFETCH
Possible Values
STOREFETCHFETCH_ONENONE

Result handling mode

Controls how query results are exposed; default FETCH returns all rows. FETCH_ONE returns the first row, STORE writes rows to Kestra storage and returns a URI, NONE leaves outputs empty.

Query filter

Optional DSL filter applied before the ES|QL query runs.

SubTypestring

Query parameters

Positional parameters for the query. Add one ? placeholder per parameter, in order. Values are substituted client-side (string interpolation) before the query is sent — this is not a server-side parameterized query. Each value is auto-typed: checked first as boolean (true/false), then integer, then long, then double, and kept as a quoted string if nothing else matches. Example: params: ["42", "true", "hello"] produces 42, true, and "hello" in the query.

Custom shard routing

Optional routing key hashed to pick the shard instead of using the document id.

First row

Populated when fetchType=FETCH_ONE; contains the first row only.

SubTypeobject

Fetched rows

Populated when fetchType=FETCH. In normal mode, each entry is a document: {field: value, ...}. In columnar mode (columnar=true), each entry is a column: {columnName: [value1, value2, ...]}, so the list length equals the number of columns, not the number of documents.

Returned row count

Number of rows included in outputs for the selected fetch type. In columnar mode (columnar=true) this reflects the number of columns, not rows.

Total rows reported

Total rows returned by the ES|QL response. In columnar mode (columnar=true) this reflects the number of columns, not rows.

Formaturi

Stored rows URI

Populated when fetchType=STORE; Kestra internal storage path to the ION file.

Unitrecords

Number of records returned

Number of ES|QL requests sent

More Plugins in this Category

Firecrawl
Firecrawl

Tasks that scrape and crawl websites into clean, LLM-ready content with Firecrawl.

Data

2 tasks

Canonical (EE)
Canonical
Enterprise

Tasks to interact with Canonical services, enabling automation and infrastructure management through API-driven workflows.

CloudInfrastructure

5 tasks