TypeSafe Evaluate
CertifiedEvaluate content with TypeSafe
TypeSafe Evaluate
Evaluate content with TypeSafe
Evaluates a state against a map of typed questions via the TypeSafe System One API and returns one structured answer per question.
type: io.kestra.plugin.typesafe.EvaluateExamples
Evaluate urgency and department
id: typesafe_evaluate
namespace: company.team
tasks:
- id: evaluate
type: io.kestra.plugin.typesafe.Evaluate
apiKey: "{{ secret('TYPESAFE_API_KEY') }}"
model: jev-latest
state: Help! My payouts have been failing for 3 days.
questions:
is_urgent:
type: NOUL
instructions: Does this convey urgency?
department:
type: CHOICE
instructions: Which team should handle this?
options:
billing: Payments, invoicing, refunds
technical: Bugs, outages, integrations
sales: Pricing, upgrades, new accounts
Rate content with a score question
id: typesafe_score
namespace: company.team
tasks:
- id: score
type: io.kestra.plugin.typesafe.Evaluate
apiKey: "{{ secret('TYPESAFE_API_KEY') }}"
state: "Customer is very frustrated with the service"
questions:
frustration:
type: SCORE
instructions: How frustrated is the customer?
levels:
- Calm
- Frustrated
- Very angry
Properties
apiKey *string
TypeSafe API key
Bearer token sent in the Authorization header; store it as a secret.
questions *object
Questions
A map of typed questions, keyed by an id you choose; answers come back under the same ids.
TypeSafe question
A typed question evaluated against a state. Use options for CHOICE, levels for SCORE, and whenTrue/whenFalse for NOUL.
Instructions
The question to evaluate. Accepts a string or structured JSON (object or array); put the question in one field and data it refers to in the others.
Score levels
Only for SCORE questions: an ordered array of level descriptions. At least 2 levels, at most 10.
Choice options
Only for CHOICE questions: a map of option name to rubric description (use null when an option needs no extra detail). Maximum 255 options.
Noul 'no' criteria
Only for NOUL questions: what a no answer (value near 0) means. Accepts a string or structured JSON.
Noul 'yes' criteria
Only for NOUL questions: what a yes answer (value near 1) means. Accepts a string or structured JSON.
state *stringobjectarray
State
The content to evaluate: a plain string for text, or structured data (object or array) for things like chat logs or records.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150baseUrl string
https://api.typesafe.aiTypeSafe base URL
Base URL of the TypeSafe API. Override it to target a mock server in tests.
model string
jev-latestModel
The model that handles the request, e.g. jev-latest. The response reports the versioned model id that answered.
Outputs
answers object
Answers
One answer per question, keyed by the same ids used in questions.
TypeSafe answer
The structured answer for one question. Only the fields matching the question type are populated.
Choice selection
Only for CHOICE answers: the highest-probability option.
Confidence
Only for CHOICE and SCORE answers: how certain the model is, between 0 and 1, derived from the probability distribution.
Score legend
Only for SCORE answers: each level number mapped back to its description.
Noul probability
Only for NOUL answers: the probability the answer is yes, on a scale from 0 (no) to 1 (yes).
Probabilities
Only for CHOICE and SCORE answers: every option (or level) mapped to its probability (floats that sum to 1).
Score value
Only for SCORE answers: the probability-weighted answer across the levels; can land between levels.
model string
Model
The versioned model id that performed the evaluation.
Metrics
input.tokens counter
tokensNumber of input tokens reported by the TypeSafe API for the request.
output.tokens counter
tokensNumber of output tokens reported by the TypeSafe API for the request.
