
TypeSafe
CertifiedEvaluate content with Jev, the decision model from TypeSafe AI.
Set your TypeSafe API key, describe typed questions once, and let Jev evaluate single states or whole datasets, with structured answers, probabilities, and confidence scores you can route on, all with retries, metrics, and ordered outputs.
TypeSafe
Evaluate content with Jev, the decision model from TypeSafe AI.
Set your TypeSafe API key, describe typed questions once, and let Jev evaluate single states or whole datasets, with structured answers, probabilities, and confidence scores you can route on, all with retries, metrics, and ordered outputs.
tasks
How to use the TypeSafe plugin
Evaluate content from Kestra flows with Jev, the decision model from TypeSafe AI. Jev answers typed questions, such as "which of these departments?", with one of the options you allow and a confidence score, instead of free text you have to parse.
Authentication
apiKey is required for the default hosted endpoint (https://api.typesafe.ai). It is optional only when baseUrl points to a compatible endpoint that accepts unauthenticated requests; the Authorization header is then not sent.
This allows the plugin to work with compatible endpoints that do not require authentication, including local TypeSafe-compatible servers such as Laya. Whether authentication can be omitted depends on the server configuration.
For authenticated endpoints, store your API key as a Kestra secret rather than hardcoding it in a flow, and read it on the task:
id: evaluate
namespace: company.team
tasks:
- id: evaluate
type: io.kestra.plugin.typesafe.Evaluate
apiKey: "{{ secret('TYPESAFE_API_KEY') }}"
model: jev-latest
state: "A customer reports a failed payment."
questions:
is_urgent:
type: NOUL
instructions: "Does this require urgent attention?"
To set apiKey once for every TypeSafe task in a namespace, use plugin defaults on Kestra 1.x, or a Policy with an Add rule on Kestra 2.x Enterprise Edition.
For a local, unauthenticated endpoint, configure baseUrl and omit apiKey:
id: evaluate
namespace: company.team
tasks:
- id: evaluate
type: io.kestra.plugin.typesafe.Evaluate
baseUrl: http://localhost: 8000
model: <model served by your endpoint>
state: "A customer reports a failed payment."
questions:
is_urgent:
type: NOUL
instructions: "Does this require urgent attention?"
The baseUrl must be reachable from the Kestra execution environment. For example, localhost refers to the Kestra container itself when Kestra runs inside Docker, not necessarily the host machine.
Token metrics
Evaluate and EvaluateBatch emit input.tokens and output.tokens from the usage the endpoint returns. A count the endpoint reports as 0 is not emitted, because some compatible servers report 0 for counts they do not track: Ollama, for example, reports output_tokens: 0 for decision models.
Model pinning
model accepts an alias such as jev-latest, which moves when TypeSafe ships a new release. Every response echoes the versioned model id that actually answered (for example jev-1.13.0), also exposed as the task model output. If you tune thresholds against a specific version, pin that versioned id in model instead of the alias.
Confidence-gated routing
CHOICE and SCORE answers carry a confidence value between 0 and 1 derived from the answer's probability distribution. Use it as a second axis in downstream tasks: act directly above your threshold and route uncertain answers to review, for example with an If task on {{ outputs.evaluate.answers.department.confidence }}.
Tasks
Evaluate evaluates one state (a string, object or array) against a map of typed questions and returns one structured answer per question, keyed by the same ids. Each question has a type (NOUL, CHOICE or SCORE) and instructions; CHOICE questions take options (at most 255), SCORE questions take levels (2 to 10), and NOUL questions optionally take whenTrue/whenFalse criteria. The output contains answers and the versioned model id; token usage is reported as input.tokens and output.tokens metrics.
EvaluateBatch evaluates every record of a dataset with the same questions. Records come from from (an inline list, a kestra:// URI, or a JSON string), are evaluated with bounded concurrency (default 5, at most 20), and results are written in input order to an ION file with one {"state", "answers"} line per record. The output contains the file uri, the count of records, and the model. If any record fails, the whole task fails with the record index in the error message.
Transient 429 and 529 responses are retried with backoff honoring the Retry-After response header; 401 and 422 responses fail fast with an actionable message.