AI Onnx
CertifiedCompute embeddings in-process with an ONNX model
AI Onnx
Compute embeddings in-process with an ONNX model
Runs a BERT-style sentence-embedding model exported to ONNX (for example all-MiniLM, BGE or E5) inside the Kestra worker with ONNX Runtime on CPU: no API key and no model server. Provide the .onnx model file and its Hugging Face tokenizer.json; modelName is only used as a label, and baseUrl, clientPem and caPem are ignored since nothing is called over the network. Embeddings only: chat completion and image generation are not supported. A loaded model stays in the worker's memory and is reused by every task that provides the same model and tokenizer files; it is unloaded after about 10 minutes without use. Each worker keeps at most 2 models loaded between embedding calls (each distinct model, tokenizer and poolingMode combination counts as one): loading another one unloads the least recently used model as soon as no running call uses it, and that model is loaded again on its next use. Administrators can change the limit with the max-loaded-models plugin configuration of io.kestra.plugin.ai.provider.Onnx; set it to at least the number of distinct models used at the same time, otherwise they keep unloading and reloading each other. Models are loaded and unloaded on a single worker thread so that reloading them reuses the same native memory: a worker loads one model at a time, so loading a large model delays the first use of another model, while calls using already loaded models are not affected. On glibc-based workers that also run other plugins allocating native memory from many threads, setting the MALLOC_ARENA_MAX=2 environment variable on the worker further limits native memory growth.
type: io.kestra.plugin.ai.provider.OnnxExamples
Ingest and search documents with all-MiniLM-L6-v2 downloaded from Hugging Face
id: onnx_rag
namespace: company.ai
tasks:
- id: model
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/onnx/model.onnx
- id: tokenizer
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/tokenizer.json
- id: ingest
type: io.kestra.plugin.ai.rag.IngestDocument
provider:
type: io.kestra.plugin.ai.provider.Onnx
modelName: all-MiniLM-L6-v2
modelUri: "{{ outputs.model.uri }}"
tokenizerUri: "{{ outputs.tokenizer.uri }}"
embeddings:
type: io.kestra.plugin.ai.embeddings.KestraKVStore
drop: true
fromDocuments:
- content: Kestra is an open-source orchestration platform for data and AI workflows.
- content: PostgreSQL is a relational database.
- id: search
type: io.kestra.plugin.ai.rag.Search
provider:
type: io.kestra.plugin.ai.provider.Onnx
modelName: all-MiniLM-L6-v2
modelUri: "{{ outputs.model.uri }}"
tokenizerUri: "{{ outputs.tokenizer.uri }}"
embeddings:
type: io.kestra.plugin.ai.embeddings.KestraKVStore
query: Which tool orchestrates workflows?
maxResults: 1
minScore: 0.3
fetchType: FETCH
Use a BGE model stored as namespace files
id: onnx_bge_search
namespace: company.ai
inputs:
- id: query
type: STRING
tasks:
- id: search
type: io.kestra.plugin.ai.rag.Search
provider:
type: io.kestra.plugin.ai.provider.Onnx
modelName: bge-small-en-v1.5
modelUri: nsfile:///models/bge-small-en-v1.5/model.onnx
tokenizerUri: nsfile:///models/bge-small-en-v1.5/tokenizer.json
poolingMode: CLS
embeddings:
type: io.kestra.plugin.ai.embeddings.KestraKVStore
query: "{{ inputs.query }}"
maxResults: 3
minScore: 0.5
fetchType: FETCH
Properties
modelUri *string
ONNX model file
URI of the .onnx model file: a namespace file (nsfile:///path), an internal storage file (kestra:///..., e.g. the output of io.kestra.plugin.core.http.Download), or a local file (file:///path, must be allowed by kestra.local-files.allowed-paths).
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
tokenizerUri *string
Tokenizer file
URI of the Hugging Face tokenizer.json published with the model, using the same schemes as modelUri. A tokenizer from another model fails, or silently produces wrong embeddings when its token ids happen to fit the model's vocabulary.
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
baseUrl string
Base URL
Custom base URL overriding the provider's default endpoint. Useful for enterprise gateways, proxies, self-hosted deployments, or test doubles such as WireMock. Defaults to the provider's public endpoint.
caPem string
CA PEM certificate content
PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.
clientPem string
Client PEM certificate content
PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.
modelName string
Model name
poolingMode string
MEANCLSMEANPooling mode
How token vectors are combined into one embedding. MEAN suits sentence-transformers models such as all-MiniLM and E5; BGE models expect CLS. A mismatch does not fail but degrades search quality. Defaults to MEAN.
