EmbeddingStoreRetriever

EmbeddingStoreRetriever

Retrieve context from an embedding store

yaml
type: io.kestra.plugin.ai.retriever.EmbeddingStoreRetriever

Examples

yaml
id: embeddingstoreretriever
type: io.kestra.plugin.ai.retriever.EmbeddingStoreRetriever
id: agent_with_rag
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      googleApiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    drop: true
    fromDocuments:
      - content: Paris is the capital of France with a population of over 2.1 million people
      - content: The Eiffel Tower is the most famous landmark in Paris at 330 meters tall

  - id: agent
    type: io.kestra.plugin.ai.agent.AIAgent
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-2.0-flash
      googleApiKey: "{{ secret('GEMINI_API_KEY') }}"
    contentRetrievers:
      - type: io.kestra.plugin.ai.retriever.EmbeddingStoreRetriever
        embeddings:
          type: io.kestra.plugin.ai.embeddings.KestraKVStore
        embeddingProvider:
          type: io.kestra.plugin.ai.provider.GoogleGemini
          modelName: gemini-embedding-exp-03-07
          googleApiKey: "{{ secret('GEMINI_API_KEY') }}"
        maxResults: 3
        minScore: 0.0
    prompt: What is the capital of France and how many people live there?

yaml
id: embeddingstoreretriever
type: io.kestra.plugin.ai.retriever.EmbeddingStoreRetriever
id: multi_store_rag
namespace: company.ai

tasks:
  - id: agent
    type: io.kestra.plugin.ai.agent.AIAgent
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-2.0-flash
      googleApiKey: "{{ secret('GEMINI_API_KEY') }}"
    contentRetrievers:
      - type: io.kestra.plugin.ai.retriever.EmbeddingStoreRetriever
        embeddings:
          type: io.kestra.plugin.ai.embeddings.Pinecone
          pineconeApiKey: "{{ secret('PINECONE_API_KEY') }}"
          index: technical-docs
        embeddingProvider:
          type: io.kestra.plugin.ai.provider.OpenAI
          googleApiKey: "{{ secret('OPENAI_API_KEY') }}"
          modelName: text-embedding-3-small
      - type: io.kestra.plugin.ai.retriever.EmbeddingStoreRetriever
        embeddings:
          type: io.kestra.plugin.ai.embeddings.Qdrant
          host: localhost
          port: 6333
          collectionName: business-docs
        embeddingProvider:
          type: io.kestra.plugin.ai.provider.GoogleGemini
          modelName: gemini-embedding-exp-03-07
          googleApiKey: "{{ secret('GEMINI_API_KEY') }}"
      - type: io.kestra.plugin.ai.retriever.TavilyWebSearch
        tavilyApiKey: "{{ secret('TAVILY_API_KEY') }}"
    prompt: What are the latest trends in data orchestration?

Properties

embeddingProvider *

Definitions

Use Amazon Bedrock models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.AmazonBedrock
      accessKeyId: "{{ secret('AWS_ACCESS_KEY') }}"
      secretAccessKey: "{{ secret('AWS_SECRET_KEY') }}"
      modelName: anthropic.claude-3-sonnet-20240229-v1:0
      thinkingBudgetTokens: 1024
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

accessKeyId*string

modelName*string

secretAccessKey*string

baseUrlstring

caPemstring

clientPemstring

modelTypestring

DefaultCOHERE

Possible Values

COHERETITAN

typeobject

Possible Values

io.kestra.plugin.ai.provider.AmazonBedrockio.kestra.plugin.langchain4j.provider.AmazonBedrock

Use Anthropic Claude models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.Anthropic
      apiKey: "{{ secret('ANTHROPIC_API_KEY') }}"
      modelName: claude-3-haiku-20240307
      thinkingEnabled: true
      thinkingBudgetTokens: 1024
      returnThinking: false
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

maxTokensintegerstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.Anthropicio.kestra.plugin.langchain4j.provider.Anthropic

Use Azure OpenAI deployments

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.AzureOpenAI
      apiKey: "{{ secret('AZURE_API_KEY') }}"
      endpoint: https://your-resource.openai.azure.com/
      modelName: anthropic.claude-3-sonnet-20240229-v1:0
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

endpoint*string

modelName*string

apiKeystring

baseUrlstring

caPemstring

clientIdstring

clientPemstring

clientSecretstring

serviceVersionstring

tenantIdstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.AzureOpenAIio.kestra.plugin.langchain4j.provider.AzureOpenAI

Use DashScope (Qwen) models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DashScope
      apiKey: "{{ secret('DASHSCOPE_API_KEY') }}"
      modelName: qwen-plus
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

Defaulthttps://dashscope-intl.aliyuncs.com/api/v1

caPemstring

clientPemstring

enableSearchbooleanstring

maxTokensintegerstring

repetitionPenaltynumberstring

typeobject

Use DeepSeek models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.DeepSeek
      apiKey: "{{ secret('DEEPSEEK_API_KEY') }}"
      modelName: deepseek-chat
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

Defaulthttps://api.deepseek.com/v1

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.DeepSeekio.kestra.plugin.langchain4j.provider.DeepSeek

Use GitHub Models via Azure AI Inference

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GitHubModels
      gitHubToken: "{{ secret('GITHUB_TOKEN') }}"
      modelName: gpt-4o-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely.
      - type: USER
        content: "{{ inputs.prompt }}"

gitHubToken*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Use Google Gemini models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      apiKey: "{{ secret('GOOGLE_API_KEY') }}"
      modelName: gemini-2.5-flash
      thinkingEnabled: true
      thinkingBudgetTokens: 1024
      returnThinking: true
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      apiKey: "{{ secret('GOOGLE_API_KEY') }}"
      modelName: gemini-2.5-flash
      clientPem: "{{ secret('CLIENT_PEM') }}"
      caPem: "{{ secret('CA_PEM') }}"
      baseUrl: "https://internal.gemini.company.com/endpoint"
      thinkingEnabled: true
      thinkingBudgetTokens: 1024
      returnThinking: true
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

embeddingModelConfiguration

io.kestra.plugin.ai.provider.GoogleGemini-EmbeddingModelConfiguration

maxRetriesintegerstring

outputDimensionalityintegerstring

taskTypestring

Possible Values

RETRIEVAL_QUERYRETRIEVAL_DOCUMENTSEMANTIC_SIMILARITYCLASSIFICATIONCLUSTERINGQUESTION_ANSWERINGFACT_VERIFICATION

timeoutstring

titleMetadataKeystring

typeobject

Possible Values

io.kestra.plugin.ai.provider.GoogleGeminiio.kestra.plugin.langchain4j.provider.GoogleGemini

Use Google Vertex AI models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.GoogleVertexAI
      endpoint: your-vertex-ai-endpoint
      location: your-google-cloud-region
      project: your-google-cloud-project-id
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

endpoint*string

location*string

modelName*string

project*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.GoogleVertexAIio.kestra.plugin.langchain4j.provider.GoogleVertexAI

Use Hugging Face Inference endpoints

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.HuggingFace
      apiKey: "{{ secret('HUGGING_FACE_API_KEY') }}"
      modelName: HuggingFaceTB/SmolLM3-3B:hf-inference
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

Defaulthttps://router.huggingface.co/v1

caPemstring

clientPemstring

typeobject

Use LocalAI OpenAI-compatible server

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.LocalAI
      modelName: gemma-3-1b-it
      baseUrl: http://localhost:8080/v1
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

baseUrl*string

modelName*string

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.LocalAIio.kestra.plugin.langchain4j.provider.LocalAI

Use Mistral models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.MistralAI
      apiKey: "{{ secret('MISTRAL_API_KEY') }}"
      modelName: mistral:7b
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.MistralAIio.kestra.plugin.langchain4j.provider.MistralAI

Use OCI Generative AI models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.OciGenAI
      region: "{{ secret('OCI_GENAI_MODEL_REGION_PROPERTY') }}"
      compartmentId: "{{ secret('OCI_GENAI_COMPARTMENT_ID_PROPERTY') }}"
      authProvider: "{{ secret('OCI_GENAI_CONFIG_PROFILE_PROPERTY') }}"
      modelName: oracle.chat.gpt-3.5
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

compartmentId*string

modelName*string

region*string

authProviderstring

baseUrlstring

caPemstring

clientPemstring

typeobject

Use local Ollama models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.Ollama
      modelName: llama3
      endpoint: http://localhost:11434
      thinkingEnabled: true
      returnThinking: true
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

endpoint*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.Ollamaio.kestra.plugin.langchain4j.provider.Ollama

Use OpenAI models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.OpenAI
      apiKey: "{{ secret('OPENAI_API_KEY') }}"
      modelName: gpt-5-mini
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

Defaulthttps://api.openai.com/v1

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.OpenAIio.kestra.plugin.langchain4j.provider.OpenAI

Use OpenRouter models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.OpenRouter
      apiKey: "{{ secret('OPENROUTER_API_KEY') }}"
      baseUrl: https://openrouter.ai/api/v1
      modelName: x-ai/grok-beta
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.OpenRouterio.kestra.plugin.langchain4j.provider.OpenRouter

Use IBM watsonx.ai models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.completion.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.WatsonxAI
      apiKey: "{{ secret('WATSONX_API_KEY') }}"
      projectId: "{{ secret('WATSONX_PROJECT_ID') }}"
      modelName: ibm/granite-3-3-8b-instruct
      baseUrl : "https://api.eu-de.dataplatform.cloud.ibm.com/wx"
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

projectId*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Use Cloudflare Workers AI models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.WorkersAI
      accountId: "{{ secret('WORKERS_AI_ACCOUNT_ID') }}"
      apiKey: "{{ secret('WORKERS_AI_API_KEY') }}"
      modelName: @cf/meta/llama-2-7b-chat-fp16
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

accountId*string

apiKey*string

modelName*string

baseUrlstring

caPemstring

clientPemstring

typeobject

Possible Values

io.kestra.plugin.ai.provider.WorkersAIio.kestra.plugin.langchain4j.provider.WorkersAI

Use ZhiPu AI models

Example

yaml
id: chat_completion
namespace: company.ai

inputs:
  - id: prompt
    type: STRING

tasks:
  - id: chat_completion
    type: io.kestra.plugin.ai.ChatCompletion
    provider:
      type: io.kestra.plugin.ai.provider.ZhiPuAI
      apiKey: "{{ secret('ZHIPU_API_KEY') }}"
      modelName: glm-4.5-flash
    messages:
      - type: SYSTEM
        content: You are a helpful assistant, answer concisely, avoid overly casual language or unnecessary verbosity.
      - type: USER
        content: "{{ inputs.prompt }}"

apiKey*string

modelName*string

baseUrlstring

Defaulthttps://open.bigmodel.cn/

caPemstring

clientPemstring

maxRetriesintegerstring

maxTokenintegerstring

stopsarray

SubTypestring

typeobject

embeddings *

Definitions

Store embeddings in Chroma

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Chroma
      baseUrl: http://localhost:8000
      collectionName: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

baseUrl*string

collectionName*string

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.Chromaio.kestra.plugin.langchain4j.embeddings.Chroma

Store embeddings in Elasticsearch

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Elasticsearch
      connection:
        hosts:
          - http://localhost:9200
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

connection*

io.kestra.plugin.ai.embeddings.Elasticsearch-ElasticsearchConnection

hosts*array

SubTypestring

Min items1

basicAuth

io.kestra.plugin.ai.embeddings.Elasticsearch-ElasticsearchConnection-BasicAuth

passwordstring

usernamestring

headersarray

SubTypestring

pathPrefixstring

strictDeprecationModebooleanstring

trustAllSslbooleanstring

indexName*string

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.Elasticsearchio.kestra.plugin.langchain4j.embeddings.Elasticsearch

Prototype embeddings in Kestra KV

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    drop: true
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

kvNamestring

Default{{ flow.id }}-embedding-store

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.KestraKVStoreio.kestra.plugin.langchain4j.embeddings.KestraKVStore

Store embeddings in MariaDB

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.MariaDB
      username: "{{ secret('MARIADB_USERNAME') }}"
      password: "{{ secret('MARIADB_PASSWORD') }}"
      databaseUrl: "{{ secret('MARIADB_DATABASE_URL') }}"
      tableName: embeddings
      fieldName: id
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

createTable*booleanstring

databaseUrl*string

fieldName*string

password*string

tableName*string

username*string

columnDefinitionsarray

SubTypestring

indexesarray

SubTypestring

metadataStorageModestring

DefaultCOLUMN_PER_KEY

typeobject

Store embeddings in Milvus

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Milvus
      # Use either `uri` or `host`/`port`:
      # For gRPC (typical): milvus://localhost:19530
      # For HTTP: http://localhost:9091
      uri: "http://localhost:19200"
      token: "{{ secret('MILVUS_TOKEN') }}"  # omit if auth is disabled
      collectionName: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

token*string

autoFlushOnDeletebooleanstring

autoFlushOnInsertbooleanstring

collectionNamestring

consistencyLevelstring

databaseNamestring

hoststring

idFieldNamestring

indexTypestring

metadataFieldNamestring

metricTypestring

passwordstring

portintegerstring

retrieveEmbeddingsOnSearchbooleanstring

textFieldNamestring

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.Milvusio.kestra.plugin.langchain4j.embeddings.Milvus

uristring

usernamestring

vectorFieldNamestring

Store embeddings in MongoDB Atlas

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.MongoDBAtlas
      username: "{{ secret('MONGODB_ATLAS_USERNAME') }}"
      password: "{{ secret('MONGODB_ATLAS_PASSWORD') }}"
      host: "{{ secret('MONGODB_ATLAS_HOST') }}"
      database: "{{ secret('MONGODB_ATLAS_DATABASE') }}"
      collectionName: embeddings
      indexName: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

collectionName*string

host*string

indexName*string

scheme*string

createIndexbooleanstring

databasestring

metadataFieldNamesarray

SubTypestring

optionsobject

passwordstring

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.MongoDBAtlasio.kestra.plugin.langchain4j.embeddings.MongoDBAtlas

usernamestring

Store embeddings with pgvector

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.PGVector
      host: localhost
      port: 5432
      user: "{{ secret('POSTGRES_USER') }}"
      password: "{{ secret('POSTGRES_PASSWORD') }}"
      database: postgres
      table: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

database*string

host*string

password*string

port*integerstring

table*string

user*string

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.PGVectorio.kestra.plugin.langchain4j.embeddings.PGVector

useIndexbooleanstring

Defaultfalse

Store embeddings in Pinecone

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Pinecone
      apiKey: "{{ secret('PINECONE_API_KEY') }}"
      cloud: AWS
      region: us-east-1
      index: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

apiKey*string

cloud*string

index*string

region*string

namespacestring

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.Pineconeio.kestra.plugin.langchain4j.embeddings.Pinecone

Store embeddings in Qdrant

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Qdrant
      apiKey: "{{ secret('QDRANT_API_KEY') }}"
      host: localhost
      port: 6334
      collectionName: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

apiKey*string

collectionName*string

host*string

port*integerstring

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.Qdrantio.kestra.plugin.langchain4j.embeddings.Qdrant

Store embeddings in Redis

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: my_api_key
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Redis
      host: localhost
      port: 6379
      indexName: embeddings
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

host*string

port*integerstring

indexNamestring

Defaultembedding-index

typeobject

Store embeddings in Alibaba Tablestore

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Tablestore
      endpoint:  "{{ secret('TABLESTORE_ENDPOINT') }}"
      instanceName:  "{{ secret('TABLESTORE_INSTANCE_NAME') }}"
      accessKeyId:  "{{ secret('TABLESTORE_ACCESS_KEY_ID') }}"
      accessKeySecret:  "{{ secret('TABLESTORE_ACCESS_KEY_SECRET') }}"
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

accessKeyId*string

accessKeySecret*string

endpoint*string

instanceName*string

metadataSchemaListarray

com.alicloud.openservices.tablestore.model.search.FieldSchema

analyzerstring

Possible Values

SingleWordMaxWordMinWordSplitFuzzy

analyzerParameter

com.alicloud.openservices.tablestore.model.search.analysis.AnalyzerParameter

dateFormatsarray

SubTypestring

enableHighlightingboolean

enableSortAndAggboolean

fieldNamestring

fieldTypestring

Possible Values

LONGDOUBLEBOOLEANKEYWORDTEXTNESTEDGEO_POINTDATEVECTORFUZZY_KEYWORDIPJSONUNKNOWN

indexboolean

indexOptionsstring

Possible Values

DOCSFREQSPOSITIONSOFFSETS

isArrayboolean

jsonTypestring

Possible Values

FLATTENNESTED

sourceFieldNamesarray

SubTypestring

storeboolean

subFieldSchemasarray

com.alicloud.openservices.tablestore.model.search.FieldSchema

analyzerstring

Possible Values

SingleWordMaxWordMinWordSplitFuzzy

analyzerParameter

dateFormatsarray

SubTypestring

enableHighlightingboolean

enableSortAndAggboolean

fieldNamestring

fieldTypestring

Possible Values

LONGDOUBLEBOOLEANKEYWORDTEXTNESTEDGEO_POINTDATEVECTORFUZZY_KEYWORDIPJSONUNKNOWN

indexboolean

indexOptionsstring

Possible Values

DOCSFREQSPOSITIONSOFFSETS

isArrayboolean

jsonTypestring

Possible Values

FLATTENNESTED

sourceFieldNamesarray

SubTypestring

storeboolean

subFieldSchemasarray

vectorOptions

com.alicloud.openservices.tablestore.model.search.vector.VectorOptions

dataTypestring

dimensioninteger

metricTypestring

Possible Values

EUCLIDEANCOSINEDOT_PRODUCT

typeobject

Store embeddings in Weaviate

Example

yaml
id: document_ingestion
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.Weaviate
      apiKey: "{{ secret('WEAVIATE_API_KEY') }}"   # omit for local/no-auth
      scheme: https                                 # http | https
      host: your-cluster-id.weaviate.network        # no protocol
      # port: 443                                   # optional; usually omit
    drop: true
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

apiKey*string

host*string

avoidDupsbooleanstring

consistencyLevelstring

Possible Values

ONEQUORUMALL

grpcPortintegerstring

metadataFieldNamestring

metadataKeysarray

SubTypestring

objectClassstring

portintegerstring

schemestring

securedGrpcbooleanstring

typeobject

Possible Values

io.kestra.plugin.ai.embeddings.Weaviateio.kestra.plugin.langchain4j.embeddings.Weaviate

useGrpcForInsertsbooleanstring

maxResults integerstring

Default3

minScore numberstring

Default0.0

Plugin Weaviate for Kestra

Integrate Google Cloud Platform services with Kestra data workflows.

Gemini Plugin for Kestra

EmbeddingStoreRetriever Retrieve context from an embedding store

More Plugins in this Category

EmbeddingStoreRetriever

Retrieve context from an embedding store