Source
yaml
id: elasticsearch
namespace: company.team
inputs:
- id: pokemon
type: STRING
defaults: jigglypuff
variables:
host: http://host.docker.internal:9200
tasks:
- id: extract
type: io.kestra.plugin.core.http.Request
uri: https://pokeapi.co/api/v2/pokemon/{{ inputs.pokemon }}
method: GET
- id: load
type: io.kestra.plugin.elasticsearch.Put
connection:
hosts:
- "{{ vars.host }}"
index: local
key: "{{ inputs.pokemon }}"
value: "{{ outputs.extract.body }}"
- id: parallel
type: io.kestra.plugin.core.flow.Parallel
tasks:
- id: search
type: io.kestra.plugin.elasticsearch.Search
connection:
hosts:
- "{{ vars.host }}"
indexes:
- local
request:
query:
term:
name:
value: "{{ inputs.pokemon }}"
- id: scroll
type: io.kestra.plugin.elasticsearch.Scroll
connection:
hosts:
- "{{ vars.host }}"
indexes:
- local
request:
query:
term:
name:
value: "{{ inputs.pokemon }}"
About this blueprint
Parallel Ingest Inputs
This flow will extract data from the Pokemon API and will load it to a given Elasticsearch index.
To validate that data has been successfully stored and indexed in Elasticsearch, you can perform search looking up that Pokemon by name.
To test this flow, you can start Elasticsearch in a Docker container:
text
docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e
"discovery.type=single-node" elasticsearch:7.10.2
For Apple Silicon macs, you can use the following command to start Elasticsearch in a Docker container:
text
docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e
"discovery.type=single-node"
docker.elastic.co/elasticsearch/elasticsearch:7.10.2