Elasticsearch Load

Elasticsearch Load

Certified

Bulk load from Kestra storage

Reads ION-serialized records from a Kestra internal storage file and indexes them in bulk. Uses the parent chunk size; set removeIdKey to keep or drop the id field after use.

yaml
type: io.kestra.plugin.elasticsearch.Load
yaml
id: elasticsearch_load
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: load
    type: io.kestra.plugin.elasticsearch.Load
    connection:
      hosts:
       - "http://localhost:9200"
    from: "{{ inputs.file }}"
    index: "my_index"
Properties

Elasticsearch connection

Connection settings shared by tasks; hosts are required.

Definitions
hosts*Requiredarray
SubTypestring
Min items1

Elasticsearch hosts

List of HTTP(S) endpoints including scheme and port, e.g. https://elasticsearch.com: 9200; at least one is required.

basicAuth

Basic authentication

Optional HTTP basic auth credentials rendered at runtime.

passwordstring

Basic auth password

Password for HTTP basic authentication.

usernamestring

Basic auth username

Username for HTTP basic authentication.

connectTimeoutstring

Connection timeout

Maximum time to wait to establish the TCP connection to an Elasticsearch host. When unset, the Elasticsearch client default applies (1 second). A value of PT0S means no timeout.

headersarray
SubTypestring

Custom HTTP headers

Headers sent on every request in Name: Value format, e.g. Authorization: Token XYZ.

pathPrefixstring

Request path prefix

Base path prepended to every Elasticsearch endpoint, e.g. /my/path. Use only when the cluster is served behind a proxy that requires a prefix; leave empty otherwise.

responseTimeoutstring

Response timeout

Maximum time to wait for a response once the request has been sent. When unset, the Elasticsearch client default applies (unbounded), so large bulk requests are not cut off. A value of PT0S means no timeout. Set a bound (e.g. PT60S) to protect against a hung cluster.

strictDeprecationModebooleanstring

Fail on warning headers

When true, any response containing Elasticsearch warning headers is treated as an error.

targetServerVersionintegerstring
Default8

Target Elasticsearch server major version

Major version used for compatibility headers (Accept and Content-Type). Set to 8 for Elasticsearch 8 clusters or 9 for Elasticsearch 9 clusters.

trustAllSslbooleanstring

Trust all SSL certificates

WARNING — SECURITY RISK: When enabled, disables BOTH TLS certificate validation (TrustAllStrategy) AND hostname verification (NoopHostnameVerifier). This makes the connection vulnerable to man-in-the-middle attacks and must never be used in production. Prefer configuring a proper truststore for self-signed certificates instead.

Source file

Kestra internal storage URI containing bulk payload; supports dynamic rendering.

The Elasticsearch index

Default1000

Bulk chunk size

Number of operations per bulk request; default 1000.

Field used as document id

Name of the field to use as _id; required when assigning ids from input rows.

Possible Values
INDEXCREATEUPDATEDELETE

Operation type

Intended bulk op type; currently only index is applied.

Defaulttrue

Remove id field from document

When true (default), drops the idKey field from the indexed document body.

Custom shard routing

Optional routing key hashed to pick the shard instead of using the document id.

Indexed document count

Total operations sent across all bulk requests.

Unitrecords

Number of records loaded

Number of bulk requests sent

Duration of bulk requests

More Plugins in this Category

VMware (EE)
VMware
Enterprise

Tasks that automate VMware vSphere operations within Kestra workflows.

Infrastructure

33 tasks

3 blueprints

Canonical (EE)
Canonical
Enterprise

Tasks to interact with Canonical services, enabling automation and infrastructure management through API-driven workflows.

CloudInfrastructure

5 tasks