PySparkSubmit icon
LoadFromGcs icon
TeamsIncomingWebhook icon
gcp icon
Schedule icon

GCS to Dataproc to BigQuery Lakehouse Refresh

Build a GCP lakehouse pipeline with Kestra. A serverless PySpark batch on Dataproc transforms GCS data and BigQuery loads a gold table, with Teams alerts.

Categories
CloudData

More Cloud Blueprints

High-Performance Computing, Run Parallel Python Workloads on AWS Batch with ECS Fargate
WorkingDirectory icon
Clone icon
TerraformCLI icon
Script icon
Docker icon
Parallel icon
+2
CloudCoreInfrastructure
Publish Application and System Events to AWS EventBridge for Event-Driven Architectures
PutEvents icon
Download icon
JsonToIon icon
Cloud
Multi-cloud deployment pipeline with Docker, ngrok, GCP Cloud Run, and Azure Container Apps
WorkingDirectory icon
Clone icon
Build icon
Switch icon
Run icon
Commands icon
+3
CloudInfrastructure

More Data Blueprints

Run Parallel ETL Ingestion with Airbyte to Sync Multiple Data Sources
Parallel icon
Sync icon
Data
Build an ETL Pipeline with Parallel Airbyte Syncs and dbt Core Transformations
Parallel icon
Sync icon
WorkingDirectory icon
Clone icon
Build icon
Docker icon
Data
Trigger an Airflow DAG
TriggerDagRun icon
Data
See How

New to Kestra?

Use blueprints to kickstart your first workflows.

Get Started