TypeSafe EvaluateBatch
CertifiedEvaluate a dataset with TypeSafe
TypeSafe EvaluateBatch
Evaluate a dataset with TypeSafe
Evaluates every record of a dataset against a map of typed questions via the TypeSafe System One API, with bounded concurrency, and writes one {"state", "answers"} line per record — in input order — to an ION file.
type: io.kestra.plugin.typesafe.EvaluateBatchExamples
Evaluate support tickets in batch
id: typesafe_batch
namespace: company.team
tasks:
- id: fetch_tickets
type: io.kestra.plugin.core.http.Download
uri: "https://example.com/tickets.csv"
- id: convert_tickets
type: io.kestra.plugin.serdes.csv.CsvToIon
from: "{{ outputs.fetch_tickets.uri }}"
- id: evaluate_tickets
type: io.kestra.plugin.typesafe.EvaluateBatch
apiKey: "{{ secret('TYPESAFE_API_KEY') }}"
model: jev-latest
from: "{{ outputs.convert_tickets.uri }}"
concurrency: 5
questions:
is_urgent:
type: NOUL
instructions: Does this convey urgency?
department:
type: CHOICE
instructions: Which team should handle this?
options:
billing: Payments, invoicing, refunds
technical: Bugs, outages, integrations
Properties
apiKey *string
TypeSafe API key
Bearer token sent in the Authorization header; store it as a secret.
from *stringarrayobject
Structured data items, either as a map, a list of map, a URI, or a JSON string.
Structured data items can be defined in the following ways:
- A single item as a map (a document).
- A list of items as a list of maps (a list of documents).
- A URI, supported schemes are
kestrafor internal storage files,filefor host local files, andnsfilefor namespace files. - A JSON String that will then be serialized either as a single item or a list of items.
Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.
questions *object
Questions
A map of typed questions, keyed by an id you choose; answers come back under the same ids.
TypeSafe question
A typed question evaluated against a state. Use options for CHOICE, levels for SCORE, and whenTrue/whenFalse for NOUL.
Instructions
The question to evaluate. Accepts a string or structured JSON (object or array); put the question in one field and data it refers to in the others.
Score levels
Only for SCORE questions: an ordered array of level descriptions. At least 2 levels, at most 10.
Choice options
Only for CHOICE questions: a map of option name to rubric description (use null when an option needs no extra detail). Maximum 255 options.
Noul 'no' criteria
Only for NOUL questions: what a no answer (value near 0) means. Accepts a string or structured JSON.
Noul 'yes' criteria
Only for NOUL questions: what a yes answer (value near 1) means. Accepts a string or structured JSON.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150baseUrl string
https://api.typesafe.aiTypeSafe base URL
Base URL of the TypeSafe API. Override it to target a mock server in tests.
concurrency integerstring
5Concurrency
Maximum number of records evaluated concurrently. Requests may complete out of order, but output lines always follow input order. Note that a persistently rate-limited (HTTP 429) call can issue up to 8 HTTP requests per record (initial attempt plus 3 retries, each transparently retried once by the underlying HTTP client), so keep this value modest to stay within the API rate limits.
model string
jev-latestModel
The model that handles the request, e.g. jev-latest. The response reports the versioned model id that answered.
Outputs
count integer
0Record count
Number of records evaluated and written to the output file.
model string
Model
The versioned model id reported by the TypeSafe API responses. When responses disagree (e.g. an alias rolled over mid-batch), this is the last evaluated record's model; per-record answers always correspond to the model that produced them. Null when the dataset is empty and no evaluation was performed.
uri string
uriOutput URI
URI of the ION file in Kestra's internal storage, with one {"state", "answers"} line per input record, in input order.
Metrics
input.tokens counter
tokensTotal number of input tokens reported by the TypeSafe API across all records.
output.tokens counter
tokensTotal number of output tokens reported by the TypeSafe API across all records.
records counter
countNumber of records evaluated and written to the output file.
