Extract Data from Couchbase and Transform It with Pandas for Analytics

Pull structured documents out of a Couchbase NoSQL database and turn them into analytics-ready data with Python and Pandas. This blueprint solves a common gap: Couchbase is great for fast operational reads and writes, but it is not where you want to run grouping, counting, and reshaping logic for reporting. Here Kestra runs a N1QL query, hands the rows to a Python script, and produces a clean CSV you can feed into a warehouse, a dashboard, or a downstream flow.

How it works

1. The query_couchbase task (io.kestra.plugin.couchbase.Query) connects to a Couchbase cluster over a connectionString and runs a N1QL query against the travel-sample.inventory.airline collection, selecting id, country, name, type, iata, and icao. With fetchType: FETCH, every matching row is returned in outputs.query_couchbase.rows.
2. The pandas task (io.kestra.plugin.scripts.python.Script) declares pandas as a dependency, loads the rows into a pandas.DataFrame, groups by country and counts airlines per country, then writes the aggregation to final.csv.
3. final.csv is captured through the task's outputFiles, making it a first-class output you can pass to other tasks or download from the execution.

What you get

• A working NoSQL-to-Python ETL example you can clone and adapt.
• A reproducible pattern using Couchbase's built-in travel-sample data.
• Query results materialized as a tabular CSV artifact per execution.
• A clean separation between extraction (N1QL) and transformation (Pandas).

Who it's for

• Data engineers bridging Couchbase and Python analytics.
• Data scientists prototyping pipelines before committing to a warehouse.
• Analysts who need recurring, reproducible extracts from operational data.

Why orchestrate this with Kestra

Couchbase has no built-in scheduler to drive cross-system pipelines, and a standalone Python script gives you no retries, no lineage, and no scheduling. With Kestra you get declarative YAML, automatic retries on transient failures, full execution logs and outputs for lineage, and event or schedule triggers so the extract runs on its own. The Python dependency (pandas) is installed per run, so the flow stays portable.

Prerequisites

• A reachable Couchbase cluster with the travel-sample bucket loaded.
• A Kestra instance that can run Python script tasks.

Secrets

This example uses inline literal credentials (username, password) and a static connectionString for the sample dataset, so no Kestra secrets are referenced. For any real cluster, replace those literals with {{ secret('COUCHBASE_USERNAME') }}, {{ secret('COUCHBASE_PASSWORD') }}, and a secret-backed connectionString.

Quick start

1. Start Couchbase locally with Docker: docker run -d --name db -p 8091-8096:8091-8096 -p 11210-11211:11210-11211 couchbase.
2. Open http://localhost:8091/, choose "Setup New Cluster", and set an admin user and password.
3. Load the travel-sample bucket from the Sample Buckets section.
4. Update connectionString, username, and password to match your cluster.
5. Add the blueprint to Kestra and run it to produce final.csv.

How to extend

• Swap the N1QL query to target other buckets, scopes, or collections.
• Replace the Pandas aggregation with joins, filters, or feature engineering.
• Add a downstream task to upload final.csv to object storage or a database.
• Attach a schedule or event trigger to run the extract automatically.

Links

• Couchbase plugin documentation