Elasticsearch Scroll

Elasticsearch Scroll

Certified

Scroll and store search results

Executes a search with a 60s scroll context and streams all hits to a Kestra internal storage file. Uses the provided request body and optional indices/routing from AbstractSearch.

yaml
type: io.kestra.plugin.elasticsearch.Scroll
yaml
id: elasticsearch_scroll
namespace: company.team

tasks:
  - id: scroll
    type: io.kestra.plugin.elasticsearch.Scroll
    connection:
      hosts:
        - "http://localhost:9200"
    indexes:
      - "my_index"
    request:
      query:
        term:
          name:
            value: 'john'
Properties

Elasticsearch connection

Connection settings shared by tasks; hosts are required.

Definitions
hosts*Requiredarray
SubTypestring
Min items1

Elasticsearch hosts

List of HTTP(S) endpoints including scheme and port, e.g. https://elasticsearch.com: 9200; at least one is required.

basicAuth

Basic authentication

Optional HTTP basic auth credentials rendered at runtime.

passwordstring

Basic auth password

Password for HTTP basic authentication.

usernamestring

Basic auth username

Username for HTTP basic authentication.

headersarray
SubTypestring

Custom HTTP headers

Headers sent on every request in Name: Value format, e.g. Authorization: Token XYZ.

pathPrefixstring

Request path prefix

Base path prepended to every Elasticsearch endpoint, e.g. /my/path. Use only when the cluster is served behind a proxy that requires a prefix; leave empty otherwise.

strictDeprecationModebooleanstring

Fail on warning headers

When true, any response containing Elasticsearch warning headers is treated as an error.

targetServerVersionintegerstring
Default8

Target Elasticsearch server major version

Major version used for compatibility headers (Accept and Content-Type). Set to 8 for Elasticsearch 8 clusters or 9 for Elasticsearch 9 clusters.

trustAllSslbooleanstring

Trust all SSL certificates

Skips certificate validation for HTTPS connections; use only with self-signed certificates in non-production.

Search request body

Elasticsearch search body as Map or JSON string; rendered before execution.

SubTypestring

Target indices

List of Elasticsearch indices to query; empty means all indices.

Reference (ref) of the pluginDefaults to apply to this task.

Custom shard routing

Optional routing key hashed to pick the shard instead of using the document id.

Fetched hit count

Total number of hits written to storage.

Formaturi

Stored hits URI

Kestra internal storage path to the Ion file containing all hits.

Unitrecords

Number of records returned

Number of scroll requests sent

Duration of scroll requests

More Plugins in this Category

NetBox (EE)
NetBox
Enterprise

Enterprise NetBox tasks to create, update, and query devices, IP addresses, and sites over the NetBox API.

Infrastructure

7 tasks

4 blueprints

Clickhouse
Clickhouse

Tasks that connect to ClickHouse via JDBC to run queries and loads.

Data

5 tasks

2 blueprints