AI Fallback

AI Fallback
Certified
Configure ordered model providers for fallback

Tries chat providers in configured order, falling back on connection failures, timeouts, and retriable provider errors such as HTTP 5xx and 429. Each provider attempt receives the full configured timeout independently, so multiple timeouts can make total duration approach N times that timeout, plus overhead; this is not guaranteed because providers differ in how they apply timeouts. It does not fall back on authentication, invalid request, guardrail, tool execution, or JSON/response-processing failures. Image and embedding operations are unsupported.

yaml
type: io.kestra.plugin.ai.provider.Fallback

Examples

AI agent with ordered provider fallback

yaml
id: ai_agent_with_fallback
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: ai_agent
    type: io.kestra.plugin.ai.agent.AIAgent
    provider:
      type: io.kestra.plugin.ai.provider.Fallback
      providers:
        - type: io.kestra.plugin.ai.provider.OpenAI
          modelName: gpt-5-mini
          apiKey: "{{ secret('OPENAI_API_KEY') }}"
          baseUrl: https://api.openai.com/v1
        - type: io.kestra.plugin.ai.provider.GoogleGemini
          modelName: gemini-2.5-flash
          apiKey: "{{ secret('GEMINI_API_KEY') }}"
    systemMessage: You are a helpful assistant. Answer clearly and concisely.
    prompt: "{{ inputs.prompt }}"

Properties

providers *

Providers

Model providers in the order they should be considered for fallback.

Definitions

Use Amazon Bedrock models

Invokes Bedrock-hosted chat/embedding models with AWS credentials. Supports standard AWS region/auth settings; ensure model IDs match your Bedrock region and account access.

Example

Chat completion with Amazon Bedrock

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.AmazonBedrock
      accessKeyId: "{{ secret('AWS_ACCESS_KEY') }}"
      secretAccessKey: "{{ secret('AWS_SECRET_KEY') }}"
      modelName: anthropic.claude-3-sonnet-20240229-v1:0
      thinkingBudgetTokens: 1024
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

accessKeyId*string

AWS Access Key ID

AWS access key ID used to sign Bedrock requests. Store it as a Kestra secret rather than inline. No default: this property is required.

secretAccessKey*string

AWS Secret Access Key

AWS secret access key paired with accessKeyId. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

Custom base URL overriding the provider's default endpoint. Useful for enterprise gateways, proxies, self-hosted deployments, or test doubles such as WireMock. Defaults to the provider's public endpoint.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

modelTypestring

DefaultCOHERE

Possible Values

COHERETITAN

Amazon Bedrock Embedding Model Type

Family of the Bedrock embedding model, which selects the request/response format used for embeddings. One of COHERE or TITAN. Defaults to COHERE. Ignored for chat and image models.

typeobject

Possible Values

io.kestra.plugin.ai.provider.AmazonBedrockio.kestra.plugin.langchain4j.provider.AmazonBedrock

Use Anthropic Claude models

Provides Claude chat models only; embeddings and images are unsupported. Enforces Anthropic rules (no seed/responseFormat). Thinking mode requires max_tokens > thinking.budget_tokens; set API key and optional base URL.

Example

Chat completion with Anthropic

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.Anthropic
      apiKey: "{{ secret('ANTHROPIC_API_KEY') }}"
      modelName: claude-3-haiku-20240307
    configuration:
      thinkingEnabled: true
      thinkingBudgetTokens: 1024
      returnThinking: false
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

Anthropic API key used to authenticate requests. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

maxTokensintegerstring

Maximum Tokens

Maximum number of tokens the model may generate in its response. Not set by default, in which case the Anthropic client default applies. When thinking is enabled, this must be greater than configuration.thinkingBudgetTokens or the task fails.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.Anthropicio.kestra.plugin.langchain4j.provider.Anthropic

Use Azure OpenAI deployments

Targets Azure-hosted OpenAI models via the resource endpoint and deployment name. Supports API key or AAD client credentials; set apiVersion when required by the deployment.

Example

Chat completion with Azure OpenAI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.AzureOpenAI
      apiKey: "{{ secret('AZURE_API_KEY') }}"
      endpoint: https://your-resource.openai.azure.com/
      modelName: gpt-4o-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

endpoint*string

API endpoint

Azure OpenAI resource endpoint, in the form https://{resource}.openai.azure.com/. No default: this property is required.

apiKeystring

API Key

Azure OpenAI API key. Provide either this key or the tenantId/clientId/clientSecret trio for Entra ID authentication; the API key takes precedence when both are set. Store it as a Kestra secret rather than inline.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientIdstring

Client ID

Microsoft Entra ID application (client) ID used for service-principal authentication. Required together with tenantId and clientSecret when apiKey is not set.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

clientSecretstring

Client secret

Microsoft Entra ID application client secret used for service-principal authentication. Required together with tenantId and clientId when apiKey is not set. Store it as a Kestra secret rather than inline.

modelNamestring

Model name

serviceVersionstring

API version

Azure OpenAI REST API version to call. Not set by default, in which case the Azure SDK's latest supported version is used.

tenantIdstring

Tenant ID

Microsoft Entra ID tenant used for service-principal authentication. Required together with clientId and clientSecret when apiKey is not set.

typeobject

Possible Values

io.kestra.plugin.ai.provider.AzureOpenAIio.kestra.plugin.langchain4j.provider.AzureOpenAI

Use DashScope (Qwen) models

Calls Alibaba Cloud DashScope for Qwen chat/embeddings/images with API key. Some params (timeouts, retries, stop, maxTokens) map directly to DashScope limits.

Example

Chat completion with DashScope (Qwen)

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DashScope
      apiKey: "{{ secret('DASHSCOPE_API_KEY') }}"
      modelName: qwen-plus
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

Alibaba Cloud DashScope API key used to authenticate requests. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Defaulthttps://dashscope-intl.aliyuncs.com/api/v1

API base URL

Base URL of the DashScope API. Use https://dashscope.aliyuncs.com/api/v1 for the China (Beijing) region and https://dashscope-intl.aliyuncs.com/api/v1 for the Singapore region. Defaults to the region inferred from the worker's system timezone.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

enableSearchbooleanstring

Enable Internet search

If true, the model may use Internet search results as reference when generating text. Defaults to false.

maxTokensintegerstring

Maximum output tokens

Maximum number of tokens returned by a single request. Not set by default, in which case the DashScope default applies.

modelNamestring

Model name

repetitionPenaltynumberstring

Repetition penalty

Penalty applied to repeated sequences during generation. Higher values reduce repetition; 1.0 means no penalty. Valid range is (0, +inf). Not set by default, in which case the DashScope default applies.

typeobject

Use DeepSeek models

Connects to DeepSeek’s OpenAI-compatible endpoint with API key and model name for chat/embedding tasks.

Example

Chat completion with DeepSeek

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DeepSeek
      apiKey: "{{ secret('DEEPSEEK_API_KEY') }}"
      modelName: deepseek-chat
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

API key used to authenticate against the provider. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Defaulthttps://api.deepseek.com/v1

API base URL

Base URL of the DeepSeek OpenAI-compatible API. Defaults to https://api.deepseek.com/v1.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.DeepSeekio.kestra.plugin.langchain4j.provider.DeepSeek

Use Docker Model Runner

Routes inference to a locally running Docker Model Runner instance via its OpenAI-compatible REST API.

Docker Model Runner is built into Docker Desktop and Docker Engine (Linux) and requires no separate setup. It exposes an OpenAI-compatible API and does not require authentication — set apiKey to any non-empty value (the default not-needed works).

Base URL variants — pick the one matching where Kestra itself runs:

Kestra in a container on Docker Desktop: http://model-runner.docker.internal/engines/v1
Kestra in a container on Docker Engine (Linux): http://172.17.0.1: 12434/engines/v1
Kestra directly on the host (default): http://localhost: 12434/engines/v1

The default suits a host installation. Most deployments run Kestra in a bridge-networked container, where localhost is the Kestra container itself rather than the Docker Model Runner host — set baseUrl explicitly in that case.

Image generation routes to the Diffusers endpoint (/engines/diffusers/v1) automatically; use a diffuser-capable model such as ai/stable-diffusion. Docker Model Runner does not advertise which models are diffuser-capable and does not reject a chat model, so passing one makes the request hang until it times out. The first image generation also downloads the Diffusers backend, which can take several minutes.

Pair this provider with io.kestra.plugin.docker.model.Pull (plugin-docker) to manage model lifecycle in the same flow.

Example

Chat completion with Docker Model Runner

yaml
id: docker_model_chat
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: pull_model
    type: io.kestra.plugin.docker.model.Pull
    model: ai/smollm2

  - id: ask
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DockerModel
      modelName: ai/smollm2
    messages:
      - type: USER
        content: "{{ inputs.prompt }}"

Chat completion (container-internal base URL)

yaml
id: docker_model_chat_container
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: ask
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DockerModel
      modelName: ai/smollm2
      baseUrl: http://model-runner.docker.internal/engines/v1
    messages:
      - type: USER
        content: "{{ inputs.prompt }}"

apiKeystring

Defaultnot-needed

API Key

Placeholder credential: Docker Model Runner requires no authentication and accepts any non-empty value. Defaults to not-needed.

baseUrlstring

Defaulthttp://localhost:12434/engines/v1

API base URL

Base URL of the Docker Model Runner OpenAI-compatible API. Pick the variant matching where Kestra itself runs: http://model-runner.docker.internal/engines/v1 for Kestra in a container on Docker Desktop, http://172.17.0.1: 12434/engines/v1 for Kestra in a container on Docker Engine (Linux), and the default http://localhost: 12434/engines/v1 for Kestra directly on the host. The model-runner.docker.internal alias exists only inside containers on Docker Desktop.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Use GitHub Models via Azure AI Inference

Calls GitHub Models through the Azure AI Inference API with a GitHub token. Supports chat and embeddings; response format options map to Azure AI Inference capabilities.

Example

Chat completion with GitHub Models

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GitHubModels
      gitHubToken: "{{ secret('GITHUB_TOKEN') }}"
      modelName: gpt-4o-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely.
      - type: USER
        content: "{{ inputs.prompt }}"

gitHubToken*string

GitHub Token

GitHub Personal Access Token (PAT) used to access GitHub Models. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Use Google Gemini models

Supports Gemini chat and embeddings (image generation is not currently supported). Tools do not support JSON Schema anyOf, and tools cannot be combined with responseFormat; configure either but not both.

Thinking models (e.g. gemini-3.5-flash) attach a thought_signature to every function-call part. This provider automatically captures those signatures (returnThinking defaults to true) and re-attaches them to the conversation history for every follow-up request (sendThinking is always enabled), preventing the 400 INVALID_ARGUMENT – Function call is missing a thought_signature error.

In addition, on Gemini 2.x models thinking is disabled by default (thinkingBudget = 0) to reduce token usage, unless thinkingEnabled: true or thinkingBudgetTokens > 0 is explicitly set. Gemini 3 models cannot have thinking turned off, so no thinking budget is sent for them by default and the model applies its own; set thinkingBudgetTokens to cap it.

Example

Chat completion with Google Gemini

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      apiKey: "{{ secret('GOOGLE_API_KEY') }}"
      modelName: gemini-3.5-flash-lite
    configuration:
      thinkingEnabled: true
      thinkingBudgetTokens: 1024
      returnThinking: true
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

Chat completion with Google Gemini with a local base URL + PEM certificates

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-3.5-flash-lite
      clientPem: "{{ secret('CLIENT_PEM') }}"
      caPem: "{{ secret('CA_PEM') }}"
      baseUrl: "https://internal.gemini.company.com/endpoint"
    configuration:
      thinkingEnabled: true
      thinkingBudgetTokens: 1024
      returnThinking: true
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKeystring

API Key

Google AI Studio API key used to authenticate requests. Store it as a Kestra secret rather than inline. Required unless certificate-based authentication is configured through clientPem, optionally with caPem.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

embeddingModelConfiguration

Embedding model configuration

Settings applied when this provider is used to generate embeddings rather than chat completions. Not set by default, in which case the Gemini client defaults apply.

io.kestra.plugin.ai.provider.GoogleGemini-EmbeddingModelConfiguration

maxRetriesintegerstring

Maximum retries

Number of times a failed embedding request is retried before the task fails. Not set by default, in which case the Gemini client default applies.

outputDimensionalityintegerstring

Output embedding size

Length the embedding vectors are truncated to, which trades a little accuracy for smaller storage. It must match the dimensionality already used in the embedding store. Not set by default (the model's full dimensionality).

taskTypestring

Possible Values

RETRIEVAL_QUERYRETRIEVAL_DOCUMENTSEMANTIC_SIMILARITYCLASSIFICATIONCLUSTERINGQUESTION_ANSWERINGFACT_VERIFICATION

Embedding task type

Downstream use the embeddings are optimized for, such as RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY or CLASSIFICATION. Use the matching pair at ingestion and query time. Not set by default, in which case the Gemini default applies.

timeoutstring

Request timeout

Maximum time to wait for each embedding request. Not set by default, in which case the Gemini client default applies.

titleMetadataKeystring

Document title metadata key

Metadata key whose value is passed to the model as the document's title, which improves retrieval quality by giving the document context. Not set by default.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.GoogleGeminiio.kestra.plugin.langchain4j.provider.GoogleGemini

Use Google Vertex AI models

Calls Vertex AI Gemini chat, embeddings, or images using project, location, and endpoint settings. Authenticates like the GCP plugin: serviceAccount when set, otherwise Application Default Credentials, optionally impersonating impersonatedServiceAccount; ensure response formats are supported by the selected model/region.

Example

Chat completion with Google Vertex AI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleVertexAI
      modelName: gemini-3.5-flash-lite
      location: your-google-cloud-region
      projectId: your-google-cloud-project-id
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

Chat completion with Google Vertex AI using a service account

yaml
id: chat_completion_service_account
namespace: company.ai

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleVertexAI
      modelName: gemini-3.5-flash-lite
      location: your-google-cloud-region
      projectId: your-google-cloud-project-id
      serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT_JSON') }}"
    messages:
      - type: USER
        content: What is the capital of France?

location*string

Project location

Google Cloud region hosting the Vertex AI model, or global to use the global endpoint aiplatform.googleapis.com. No default: this property is required for chat models.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

endpointstring

Endpoint URL

Vertex AI API endpoint for image and embedding models. Not set by default, in which case the endpoint is derived from location (aiplatform.googleapis.com: 443 for global). Must not be set for chat models, which always use Gemini.

impersonatedServiceAccountstring

Service account to impersonate

Email of the service account to impersonate. Not supported for image models.

modelNamestring

Model name

projectstring

Project ID (deprecated)

Deprecated, use projectId instead. Only used when projectId is not set.

projectIdstring

Project ID

Google Cloud project ID that owns the Vertex AI resources. When not set, it is taken from the service account credentials.

scopesarray

SubTypestring

Default["https://www.googleapis.com/auth/cloud-platform"]

OAuth scopes

OAuth scopes requested for the credentials. Defaults to https://www.googleapis.com/auth/cloud-platform. Not supported for image models, which always use Application Default Credentials with their default scopes.

serviceAccountstring

Service account JSON key

Content of a Google Cloud service account JSON key. When not set, Application Default Credentials are used. Not supported for image models, which always use Application Default Credentials.

typeobject

Possible Values

io.kestra.plugin.ai.provider.GoogleVertexAIio.kestra.plugin.langchain4j.provider.GoogleVertexAI

Use Hugging Face Inference endpoints

Routes requests to Hugging Face Inference Endpoints via the OpenAI-compatible gateway (default router.huggingface.co). Requires an API token and deployment model name.

Example

Chat completion with HuggingFace

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.HuggingFace
      apiKey: "{{ secret('HUGGING_FACE_API_KEY') }}"
      modelName: HuggingFaceTB/SmolLM3-3B:hf-inference
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

API key used to authenticate against the provider. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Defaulthttps://router.huggingface.co/v1

API base URL

Base URL of the Hugging Face router's OpenAI-compatible API. Defaults to https://router.huggingface.co/v1.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Use Langdock models

Connects to Langdock's Completion API, which exposes OpenAI/Azure OpenAI-backed models on the OPENAI route and Claude models on the ANTHROPIC route. Set modelFamily to match the modelName you use: Claude models are only reachable when modelFamily is ANTHROPIC. Use the langdock.ListModels task with the matching family to discover valid modelName values.

Embeddings always go through the OpenAI route (only text-embedding-ada-002 is supported there). If your key is refused for embeddings, use a workspace API key with the Embedding API scope. Image generation is not offered by the Completion API.

For a dedicated deployment, set baseUrl to the route root that matches the operation you are using this provider for, e.g. https://acme.langdock.com/api/public/openai/eu/v1 for chat/embeddings on the OpenAI route, or https://acme.langdock.com/api/public/anthropic/eu/v1/ for chat on the Anthropic route; it then takes precedence over region.

Example

Chat completion with a Langdock-hosted OpenAI model

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.Langdock
      apiKey: "{{ secret('LANGDOCK_API_KEY') }}"
      modelFamily: OPENAI
      modelName: gpt-5.4-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

AI agent using a Langdock-hosted Claude model with a Kestra tool

yaml
id: agent_with_tool
namespace: company.ai

inputs:
  - id: prompt
    type: STRING
    defaults: "Log the message 'Hello from Langdock!'"

tasks:
  - id: agent
    type: io.kestra.plugin.ai.agent.AIAgent
    provider:
      type: io.kestra.plugin.ai.provider.Langdock
      apiKey: "{{ secret('LANGDOCK_API_KEY') }}"
      modelFamily: ANTHROPIC
      modelName: claude-sonnet-4-6-default
    prompt: "{{ inputs.prompt }}"
    tools:
      - type: io.kestra.plugin.ai.tool.KestraTask
        tasks:
          - id: log
            type: io.kestra.plugin.core.log.Log
            message: "..."

Ingest documents into a KV embedding store using Langdock embeddings

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.Langdock
      apiKey: "{{ secret('LANGDOCK_WORKSPACE_API_KEY') }}"
      modelName: text-embedding-ada-002
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    drop: true
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/README.md

apiKey*string

API Key

Langdock API key used to authenticate requests. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelFamilystring

DefaultOPENAI

Possible Values

OPENAIANTHROPIC

Model family

Langdock Completion API route serving the request: OPENAI for OpenAI and Azure OpenAI-backed models, or ANTHROPIC, which is required to reach Claude models. Defaults to OPENAI. Ignored for embeddings, which always use the OpenAI route.

modelNamestring

Model name

regionstring

DefaultEU

Possible Values

EUUS

Region

Langdock region that serves the request: EU or US. Defaults to EU. Ignored when baseUrl points at a dedicated deployment.

typeobject

Use LocalAI OpenAI-compatible server

Targets a self-hosted LocalAI instance via its OpenAI-compatible API for chat/embeddings/images. Set baseUrl if your server is not on the default.

Example

Chat completion with LocalAI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.LocalAI
      modelName: gemma-3-1b-it
      baseUrl: http://localhost:8080/v1
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

baseUrl*string

API base URL

Base URL of the LocalAI server's OpenAI-compatible API. No default: this property is required.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.LocalAIio.kestra.plugin.langchain4j.provider.LocalAI

Use Mistral models

Calls Mistral chat/embedding APIs with an API key. topK is not supported; chat configuration must respect model limits.

Example

Chat completion with Mistral AI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.MistralAI
      apiKey: "{{ secret('MISTRAL_API_KEY') }}"
      modelName: mistral-small-latest
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

Mistral AI API key used to authenticate requests. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.MistralAIio.kestra.plugin.langchain4j.provider.MistralAI

Use OCI Generative AI models

Calls Oracle Cloud GenAI chat/embedding models with compartment OCID and region. Auth is handled via OCI SDK provider (config file or instance principals). Ensure the model is permitted in the chosen compartment.

Example

Chat completion with OciGenAI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.OciGenAI
      region: "{{ secret('OCI_GENAI_MODEL_REGION_PROPERTY') }}"
      compartmentId: "{{ secret('OCI_GENAI_COMPARTMENT_ID_PROPERTY') }}"
      authProvider: "{{ secret('OCI_GENAI_CONFIG_PROFILE_PROPERTY') }}"
      modelName: cohere.command-r-plus
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

compartmentId*string

Compartment OCID

OCID of the OCI compartment holding the generative AI model. No default: this property is required.

region*string

OCI region

OCI region the client connects to, which must offer the OCI Generative AI service. No default: this property is required.

authProviderstring

OCI config profile name

Name of the profile in your OCI config file used to authenticate the SDK client. Defaults to DEFAULT.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Use local Ollama models

Calls an Ollama server for chat/embeddings using the given endpoint and model name. Ideal for self-hosted/local models; ensure the Ollama daemon is reachable.

Example

Chat completion with Ollama

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.Ollama
      modelName: llama3
      endpoint: http://localhost:11434
    configuration:
      thinkingEnabled: true
      returnThinking: true
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

endpoint*string

Model endpoint

Base URL of the Ollama server exposing the model. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.Ollamaio.kestra.plugin.langchain4j.provider.Ollama

Use OpenAI models

Connects to OpenAI-compatible endpoints (defaults to api.openai.com) for chat, embeddings, or images. Requires API key; override baseUrl for Azure-compatible or proxy setups.

Example

Chat completion with OpenAI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.OpenAI
      apiKey: "{{ secret('OPENAI_API_KEY') }}"
      modelName: gpt-5-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

API key used to authenticate against the provider. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Defaulthttps://api.openai.com/v1

API base URL

Base URL of the OpenAI-compatible API. Override it to target Azure OpenAI, a proxy, or a self-hosted gateway. Defaults to https://api.openai.com/v1.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.OpenAIio.kestra.plugin.langchain4j.provider.OpenAI

Use OpenRouter models

Routes requests through OpenRouter’s multi-model API using your API key. Supports chat/embedding/image models exposed by OpenRouter; honor provider-specific safety/usage limits.

Example

Chat completion with OpenRouter

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.OpenRouter
      apiKey: "{{ secret('OPENROUTER_API_KEY') }}"
      baseUrl: https://openrouter.ai/api/v1
      modelName: openai/gpt-4o-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

OpenRouter API key used to authenticate requests. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.OpenRouterio.kestra.plugin.langchain4j.provider.OpenRouter

Use IBM watsonx.ai models

Calls IBM watsonx.ai chat/embedding endpoints with API key and project ID. Ensure the selected model ID is available in the configured project.

Example

Chat completion with Watsonx AI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.WatsonxAI
      apiKey: "{{ secret('WATSONX_API_KEY') }}"
      projectId: "{{ secret('WATSONX_PROJECT_ID') }}"
      modelName: ibm/granite-3-3-8b-instruct
      baseUrl : "https://api.eu-de.dataplatform.cloud.ibm.com/wx"
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

IBM Cloud API key used to authenticate against watsonx.ai. Store it as a Kestra secret rather than inline. No default: this property is required.

projectId*string

Project ID

Identifier of the watsonx.ai project the model runs under. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Use Cloudflare Workers AI models

Invokes Workers AI chat, embedding, and image models using account ID and API key. Ensure the selected model is available in your account/region.

Example

Chat completion with WorkersAI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.WorkersAI
      accountId: "{{ secret('WORKERS_AI_ACCOUNT_ID') }}"
      apiKey: "{{ secret('WORKERS_AI_API_KEY') }}"
      modelName: "@cf/meta/llama-2-7b-chat-fp16"
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

accountId*string

Account Identifier

Cloudflare account ID that owns the Workers AI deployment. No default: this property is required.

apiKey*string

API Key

Cloudflare API token with Workers AI access. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Base URL

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelNamestring

Model name

typeobject

Possible Values

io.kestra.plugin.ai.provider.WorkersAIio.kestra.plugin.langchain4j.provider.WorkersAI

Use ZhiPu AI models

Calls ZhiPu’s OpenAI-compatible chat/embedding/image APIs with API key and model name. Supports stop tokens, retry count, and max tokens per request.

Example

Chat completion with ZhiPu AI

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.ZhiPuAI
      apiKey: "{{ secret('ZHIPU_API_KEY') }}"
      modelName: glm-4.5-flash
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

API Key

ZhiPu AI API key used to authenticate requests. Store it as a Kestra secret rather than inline. No default: this property is required.

baseUrlstring

Defaulthttps://open.bigmodel.cn/

API base URL

Base URL of the ZhiPu AI API. Defaults to https://open.bigmodel.cn/.

caPemstring

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPemstring

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

maxRetriesintegerstring

Maximum retries

Number of times a failed request is retried before the task fails. Defaults to 3.

maxTokenintegerstring

Maximum output tokens

Maximum number of tokens returned by a single request. Defaults to 512.

modelNamestring

Model name

stopsarray

SubTypestring

Stop sequences

Strings that stop generation as soon as the model is about to produce one of them. The stop sequence itself is not included in the output. Not set by default (no stop sequence).

typeobject

baseUrl string

Base URL

caPem string

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPem string

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

modelName string

Model name

Google Cloud

Execute tasks and orchestrate workflows across Google Cloud Platform services. Centralize authentication and resource management for seamless GCP integration.

CloudDataInfrastructureAI

Databricks

Databricks plugin for Kestra

DataAI

AI FallbackCertifiedConfigure ordered model providers for fallback

More Plugins in this Category

Google Cloud

Databricks

AI Fallback
Certified
Configure ordered model providers for fallback