Home
Blueprints

GCS to Dataproc to BigQuery Lakehouse Refresh

Build a GCP lakehouse pipeline with Kestra. A serverless PySpark batch on Dataproc transforms GCS data and BigQuery loads a gold table, with Teams alerts.

More Cloud Blueprints

High-Performance Computing, Run Parallel Python Workloads on AWS Batch with ECS Fargate

Provision AWS Batch on ECS Fargate with Terraform, then run parallel Python workloads on it from one Kestra flow.

CloudCoreInfrastructure

Enterprise Edition

Publish Application and System Events to AWS EventBridge for Event-Driven Architectures

Publish custom application and system events to AWS EventBridge using inline definitions or bulk JSON payloads.

Cloud

Multi-cloud deployment pipeline with Docker, ngrok, GCP Cloud Run, and Azure Container Apps

Build a Docker image from a GitHub repo and deploy it to local, GCP Cloud Run, or Azure Container Instances from one declarative flow.

CloudInfrastructure

Autoscale EMR Serverless Spark on a CloudWatch Backlog

Watch a CloudWatch backlog metric and fire an EMR Serverless Spark job only when work is pending.

CloudData

Ansible Scalability, Deploy Configuration Files to Multiple Servers at Scale

Provision AWS EC2 servers and deploy configuration files across all of them at scale with Ansible.

CloudInfrastructure

Stream Kestra audit logs from Kafka to BigQuery

Stream Kestra audit logs from a Kafka topic into Google BigQuery for analytics, monitoring, and compliance reporting.

Cloud

More Data Blueprints

Run Parallel ETL Ingestion with Airbyte to Sync Multiple Data Sources

Trigger multiple Airbyte connection syncs in parallel from a single Kestra flow to cut total ingestion time.

Data

AI-Powered Inventory Demand Forecasting, Auto-Generate Purchase Orders with OpenAI

Forecast 7-day demand with OpenAI GPT-4o, auto-generate purchase orders for stockout-risk SKUs, and log them to PostgreSQL every 6 hours.

AIBusinessData

Trigger an Airflow DAG

Trigger an Apache Airflow DAG from Kestra and wait for it to finish.

Data

Generate and Execute SQL Queries from Natural Language Using OpenAI GPT-4o

Turn plain-English questions into PostgreSQL queries with OpenAI GPT-4o, grounded on schema only so no row data ever leaves your database.

AIData

Build an ETL Pipeline with Airbyte Cloud and dbt Cloud for SaaS Analytics

Run Airbyte Cloud syncs in parallel, then trigger a dbt Cloud transformation job once the data lands.

Data

AI Document Data Extraction: Parse PDFs and Web Pages into Structured JSON with OpenAI

Turn any PDF, web page, or text document into structured JSON with OpenAI GPT-4o, then store the results in PostgreSQL and Slack.

AIData

New to Kestra?

Use blueprints to kickstart your first workflows.

Get Started