AI DockerModel

AI DockerModel
Certified
Use Docker Model Runner

Routes inference to a locally running Docker Model Runner instance via its OpenAI-compatible REST API.

Docker Model Runner is built into Docker Desktop and Docker Engine (Linux) and requires no separate setup. It exposes an OpenAI-compatible API and does not require authentication — set apiKey to any non-empty value (the default not-needed works).

Base URL variants — pick the one matching where Kestra itself runs:

Kestra in a container on Docker Desktop: http://model-runner.docker.internal/engines/v1
Kestra in a container on Docker Engine (Linux): http://172.17.0.1: 12434/engines/v1
Kestra directly on the host (default): http://localhost: 12434/engines/v1

The default suits a host installation. Most deployments run Kestra in a bridge-networked container, where localhost is the Kestra container itself rather than the Docker Model Runner host — set baseUrl explicitly in that case.

Image generation routes to the Diffusers endpoint (/engines/diffusers/v1) automatically; use a diffuser-capable model such as ai/stable-diffusion. Docker Model Runner does not advertise which models are diffuser-capable and does not reject a chat model, so passing one makes the request hang until it times out. The first image generation also downloads the Diffusers backend, which can take several minutes.

Pair this provider with io.kestra.plugin.docker.model.Pull (plugin-docker) to manage model lifecycle in the same flow.

yaml
type: io.kestra.plugin.ai.provider.DockerModel

Examples

Chat completion with Docker Model Runner

yaml
id: docker_model_chat
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: pull_model
    type: io.kestra.plugin.docker.model.Pull
    model: ai/smollm2

  - id: ask
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DockerModel
      modelName: ai/smollm2
    messages:
      - type: USER
        content: "{{ inputs.prompt }}"

Chat completion (container-internal base URL)

yaml
id: docker_model_chat_container
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: ask
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DockerModel
      modelName: ai/smollm2
      baseUrl: http://model-runner.docker.internal/engines/v1
    messages:
      - type: USER
        content: "{{ inputs.prompt }}"

Properties

modelName *string

Model name

Identifier of the model to call, as named by the provider. Valid values depend on the provider and on whether the model is used for chat, embeddings, or image generation; see the provider's model catalog. No default: this property is required.

apiKey string

Defaultnot-needed

API Key

Placeholder credential: Docker Model Runner requires no authentication and accepts any non-empty value. Defaults to not-needed.

baseUrl string

Defaulthttp://localhost:12434/engines/v1

API base URL

Base URL of the Docker Model Runner OpenAI-compatible API. Pick the variant matching where Kestra itself runs: http://model-runner.docker.internal/engines/v1 for Kestra in a container on Docker Desktop, http://172.17.0.1: 12434/engines/v1 for Kestra in a container on Docker Engine (Linux), and the default http://localhost: 12434/engines/v1 for Kestra directly on the host. The model-runner.docker.internal alias exists only inside containers on Docker Desktop.

caPem string

CA PEM certificate content

PEM-encoded certificate authority chain, as text, used to verify the TLS certificate presented by a custom endpoint. Not set by default, in which case the JVM's default trust store is used.

clientPem string

Client PEM certificate content

PEM-encoded client certificate and private key, as text, used for mutual-TLS authentication against enterprise AI endpoints. Not set by default, in which case the default HTTP client is used.

Huggingface

Tasks that call Hugging Face Inference APIs.

Gemini

Tasks that call Google Gemini models for text, chat, structured outputs, multimodal prompts, and video generation.

AI DockerModelCertifiedUse Docker Model Runner

AI DockerModel

More Plugins in this Category

Huggingface

Gemini

AI DockerModel
Certified
Use Docker Model Runner