Load
type: "io.kestra.plugin.elasticsearch.Load"
Bulk load documents in elasticsearch using Kestra Internal Storage file
Examples
id: "load"
type: "io.kestra.plugin.elasticsearch.Load"
connection:
hosts:
- "http://localhost:9200"
from: "{{ inputs.file }}"
index: "my_index"
Properties
connection
- Type: ElasticsearchConnection
- Dynamic: ❓
- Required: ✔️
The connection properties.
from
- Type: string
- Dynamic: ✔️
- Required: ✔️
The source file
index
- Type: string
- Dynamic: ✔️
- Required: ✔️
The elasticsearch indice
chunk
- Type: integer
- Dynamic: ✔️
- Required: ❌
The size of chunk for every bulk request
idKey
- Type: string
- Dynamic: ✔️
- Required: ❌
Use this key as id.
opType
- Type: string
- Dynamic: ❌
- Required: ❌
- Possible Values:
INDEX
CREATE
UPDATE
DELETE
Sets the type of operation to perform.
removeIdKey
- Type: boolean
- Dynamic: ✔️
- Required: ❌
- Default:
true
Remove idKey from the final document
routing
- Type: string
- Dynamic: ✔️
- Required: ❌
Controls the shard routing of the request.
Using this value to hash the shard and not the id.
Outputs
size
- Type: integer
The size of the rows fetch
Definitions
ElasticsearchConnection
hosts
- Type: array
- SubType: string
- Dynamic: ✔️
- Required: ✔️
- Min items:
1
List of http elasticsearch servers
Must be an uri like https://elasticsearch.com:9200
with scheme and port
basicAuth
- Type: BasicAuth
- Dynamic: ❌
- Required: ❌
Basic Auth configuration
headers
- Type: array
- SubType: string
- Dynamic: ✔️
- Required: ❌
List of http headers to be send on every request
Must be an string with key value separated with :
, ex: Authorization: Token XYZ
pathPrefix
- Type: string
- Dynamic: ✔️
- Required: ❌
Sets the path's prefix for every request used by the http client.
For example, if this is set to /my/path
, then any client request will become /my/path/
+ endpoint.
In essence, every request's endpoint is prefixed by this pathPrefix
.
The path prefix is useful for when Elasticsearch is behind a proxy that provides a base path or a proxy that requires all paths to start with '/'; it is not intended for other purposes and it should not be supplied in other scenarios.
strictDeprecationMode
- Type: boolean
- Dynamic: ❌
- Required: ❌
Whether the REST client should return any response containing at leas one warning header as a failure.
trustAllSsl
- Type: boolean
- Dynamic: ❌
- Required: ❌
Trust all SSL CA certificate
Use this if the server is using a self signed ssl certificate.
BasicAuth
password
- Type: string
- Dynamic: ✔️
- Required: ❌
Basic Auth password
username
- Type: string
- Dynamic: ✔️
- Required: ❌
Basic Auth username