Bootstrap a Pinecone RAG Index as Code with a Stats Readback

Vector indexes created by hand in the Pinecone console drift: staging gets 768 dimensions, production gets 1024, and nobody remembers which region the dev index landed in. This blueprint makes the index spec a versioned artifact. io.kestra.plugin.pinecone.CreateIndex provisions a serverless index with the dimension, cloud, and region fixed in YAML, io.kestra.plugin.pinecone.DescribeIndexStats reads the index back to prove the data plane answers, and Slack gets a confirmation with the status, host, and starting vector count.

How it works

1. create_index (io.kestra.plugin.pinecone.CreateIndex) creates a serverless index named by the index_name input with dimension: 1024, cloud: aws, and region: us-east-1. The task outputs the index name, host, and status.
2. wait_for_init sleeps 15 seconds so the fresh index has a moment to leave its initializing state.
3. readback_stats (io.kestra.plugin.pinecone.DescribeIndexStats) queries the new index and returns totalVectorCount, which is zero for a fresh index, proving the index accepts data-plane calls.
4. notify posts the index name, status, host, and starting vector count to Slack.
5. The errors block posts a distinct Slack alert when any step fails, warning that the index may exist in a partial state.

What you get

• An index spec that lives in version control instead of someone's memory of console clicks.
• A readback step that verifies the index actually serves requests before anyone builds on it.
• Identical environments on demand: rerun the flow with a different index_name for staging, dev, or a migration target.
• A Slack record of every environment bootstrap with the exact host and status.

Who it's for

• Platform teams standing up RAG infrastructure across dev, staging, and production.
• ML engineers who want vector index provisioning reviewed in a pull request like any other infrastructure.
• Teams migrating embedding models who need to spin up a new index with a different dimension safely.

Why orchestrate this with Kestra

A console click leaves no trace and no guarantee the next environment matches. Kestra turns the index spec into declarative YAML with an execution history: every bootstrap is auditable, the readback catches indexes that were created but never became ready, and the same flow parameterizes cleanly across environments through a single input.

Prerequisites

• A Pinecone account with serverless index quota on AWS us-east-1, or adjust cloud and region.
• The dimension must match your embedding model; 1024 fits models like Cohere embed-english-v3.0.
• A Slack incoming webhook for confirmations and alerts.

Secrets

• PINECONE_API_KEY: Pinecone API key with index creation permission.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the PINECONE_API_KEY and SLACK_WEBHOOK_URL secrets to your Kestra namespace.
2. Execute the flow and check the Slack message reports status and a zero starting vector count.
3. Rerun with a different index_name value to bootstrap the next environment.

How to extend

• Set deletionProtection on the create task to guard production indexes against accidental deletes.
• Chain the embeddings ingestion blueprint after the bootstrap so a new environment loads its corpus immediately.
• Add the index stats monitor blueprint on a schedule to watch the index once it carries data.
• Parameterize dimension as an input when different corpora use different embedding models.

Links

• Pinecone plugin documentation
• Pinecone documentation