Use OpenSearch as a Config Store with Put and Get Roundtrip

Every team ends up with small reference documents that many flows need: batch sizes, environment targets, feature switches, ownership records. This blueprint shows the simplest durable home for them, an OpenSearch index used as a key-value config store. io.kestra.plugin.opensearch.Put writes a JSON document under a fixed key, io.kestra.plugin.opensearch.Request reads it back by that key, and a log task consumes individual fields from the returned document.

How it works

1. put_config (io.kestra.plugin.opensearch.Put) indexes a small map of settings into the flow-config index under the key ingestion-settings. Writing the same key again overwrites the document, so the store always holds the latest version.
2. get_config (io.kestra.plugin.opensearch.Request) fetches the document by index and key. The document is exposed as {{ outputs.get_config.row }}, with each field addressable directly.
3. log_owner logs {{ outputs.get_config.response._source.owner }} and {{ outputs.get_config.response._source.batch_size }}, the same pattern any downstream task would use to consume stored settings.
4. The errors block posts a Discord alert when either the write or the read fails, naming the flow and execution.

What you get

• A key-value config store built on infrastructure you already run, no extra database to operate.
• Documents addressable by human-readable keys instead of generated ids.
• Stored fields flowing straight into downstream task expressions as scalars.
• A failure alert so dependent flows never silently read stale or missing settings.

Who it's for

• Platform teams who want shared flow settings outside of hardcoded YAML values.
• Data engineers already running OpenSearch for logs or search who need a small reference store.
• Anyone replacing config files scattered across repositories with one queryable index.

Why orchestrate this with Kestra

A config store is only useful when reads and writes are observable. Kestra gives the roundtrip an execution history, per-task logs, and outputs that flow into any downstream task, so you can see exactly which execution wrote which settings and which flows consumed them. Splitting the Put into its own flow later gives you an audited write path with a full change trail.

Prerequisites

• A reachable OpenSearch cluster and credentials allowed to write to the flow-config index.
• A Discord incoming webhook for failure alerts.

Secrets

• OPENSEARCH_HOST: OpenSearch endpoint, for example https://opensearch.example.com:9200.
• OPENSEARCH_USERNAME: OpenSearch username.
• OPENSEARCH_PASSWORD: OpenSearch password.
• DISCORD_WEBHOOK_URL: Discord incoming webhook URL.

Quick start

1. Add the four secrets to your Kestra namespace.
2. Execute the flow and check the log line shows the owner and batch size read back from OpenSearch.
3. Replace the sample value map with your real settings and rename the index and key to match your conventions.

How to extend

• Split Put and Get into separate flows so one governed flow owns writes and many flows read.
• Add more documents under different keys, one per pipeline or environment.
• Feed {{ outputs.get_config.response._source.batch_size }} into task properties of a real ingestion flow.
• Swap the log task for io.kestra.plugin.core.flow.If to branch on a stored feature switch.

Links

• OpenSearch plugin documentation
• OpenSearch documentation