scroll
Get all documents from a search request and store it as a Kestra Internal Storage file.
Get all documents from a search request and store it as a Kestra Internal Storage file.
Get all documents from a search request and store it as a Kestra Internal Storage file.
type: "io.kestra.plugin.elasticsearch.scroll"Examples
id: elasticsearch_scroll
namespace: company.team
tasks:
- id: scroll
type: io.kestra.plugin.elasticsearch.Scroll
connection:
hosts:
- "http://localhost:9200"
indexes:
- "my_index"
request:
query:
term:
name:
value: 'john'
Properties
connection*RequiredNon-dynamic
The connection properties.
io.kestra.plugin.elasticsearch.ElasticsearchConnection
1List of HTTP ElasticSearch servers.
Must be an URI like https://elasticsearch.com: 9200 with scheme and port.
Basic auth configuration.
io.kestra.plugin.elasticsearch.ElasticsearchConnection-BasicAuth
Basic auth password.
Basic auth username.
List of HTTP headers to be send on every request.
Must be a string with key value separated with : , ex: Authorization: Token XYZ.
Sets the path's prefix for every request used by the HTTP client.
For example, if this is set to /my/path, then any client request will become /my/path/ + endpoint.
In essence, every request's endpoint is prefixed by this pathPrefix.
The path prefix is useful for when ElasticSearch is behind a proxy that provides a base path or a proxy that requires all paths to start with '/'; it is not intended for other purposes and it should not be supplied in other scenarios.
Whether the REST client should return any response containing at least one warning header as a failure.
Trust all SSL CA certificates.
Use this if the server is using a self signed SSL certificate.
request*Requiredobject
The ElasticSearch value.
Can be a JSON string. In this case, the contentType will be used or a raw Map.
indexesarray
The ElasticSearch indices.
Default to all indices.
routingstring
Controls the shard routing of the request.
Using this value to hash the shard and not the id.
Outputs
sizeinteger
The size of the rows fetch
uristring
uriThe uri of store result
Metrics
recordscounter
recordsNumber of records returned
requests.countcounter
Number of scroll requests sent
requests.durationtimer
Duration of scroll requests
