AI DockerModel

AI DockerModel

Certified

Use Docker Model Runner

Routes inference to a locally running Docker Model Runner instance via its OpenAI-compatible REST API.

Docker Model Runner is built into Docker Desktop and Docker Engine (Linux) and requires no separate setup. It exposes an OpenAI-compatible API and does not require authentication — set apiKey to any non-empty value (the default not-needed works).

Base URL variants — pick the one matching where Kestra itself runs:

  • Kestra in a container on Docker Desktop: http://model-runner.docker.internal/engines/v1
  • Kestra in a container on Docker Engine (Linux): http://172.17.0.1: 12434/engines/v1
  • Kestra directly on the host (default): http://localhost: 12434/engines/v1

The default suits a host installation. Most deployments run Kestra in a bridge-networked container, where localhost is the Kestra container itself rather than the Docker Model Runner host — set baseUrl explicitly in that case.

Image generation routes to the Diffusers endpoint (/engines/diffusers/v1) automatically; use a diffuser-capable model such as ai/stable-diffusion. Docker Model Runner does not advertise which models are diffuser-capable and does not reject a chat model, so passing one makes the request hang until it times out. The first image generation also downloads the Diffusers backend, which can take several minutes.

Pair this provider with io.kestra.plugin.docker.model.Pull (plugin-docker) to manage model lifecycle in the same flow.

yaml
type: io.kestra.plugin.ai.provider.DockerModel

Chat completion with Docker Model Runner

yaml
id: docker_model_chat
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: pull_model
    type: io.kestra.plugin.docker.model.Pull
    model: ai/smollm2

  - id: ask
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DockerModel
      modelName: ai/smollm2
    messages:
      - type: USER
        content: "{{ inputs.prompt }}"

Chat completion (container-internal base URL)

yaml
id: docker_model_chat_container
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: ask
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DockerModel
      modelName: ai/smollm2
      baseUrl: http://model-runner.docker.internal/engines/v1
    messages:
      - type: USER
        content: "{{ inputs.prompt }}"
Properties

Model name

Defaultnot-needed

API Key

Docker Model Runner does not require authentication. Any non-empty value is accepted; defaults to not-needed.

Defaulthttp://localhost:12434/engines/v1

API base URL

Base URL for the Docker Model Runner OpenAI-compatible API. Pick the variant matching where Kestra itself runs:

  • Kestra in a container on Docker Desktop: http://model-runner.docker.internal/engines/v1
  • Kestra in a container on Docker Engine (Linux): http://172.17.0.1: 12434/engines/v1
  • Kestra directly on the host (default): http://localhost: 12434/engines/v1

The model-runner.docker.internal alias exists only inside containers on Docker Desktop.

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

More Plugins in this Category

Mistral
Mistral

Tasks and triggers for Mistral AI: chat completions and workflow orchestration.

AI

3 tasks

DeepSeek
DeepSeek

Tasks that call DeepSeek chat models for conversational or schema-guided JSON responses.

AI

1 tasks

1 blueprints