Query icon
If icon
PySparkSubmit icon
DiscordIncomingWebhook icon
gcp icon
Schedule icon

Gate a Dataproc PySpark Batch on Data Readiness

Orchestrate Dataproc Serverless with Kestra. Check BigQuery for fresh source rows, then submit a PySpark batch only when the data has landed.

Categories
CloudData

More Cloud Blueprints

Publish Application and System Events to AWS EventBridge for Event-Driven Architectures
PutEvents icon
Download icon
JsonToIon icon
Cloud
Stream JSON and Application Events to AWS Kinesis Data Streams at Scale
PutRecords icon
Download icon
JsonToIon icon
Cloud
Extract a CSV file via HTTP API and upload it to S3 by using scheduled
Download icon
ArchiveDecompress icon
Upload icon
Schedule icon
CloudCore

More Data Blueprints

Trigger an Airflow DAG
TriggerDagRun icon
Data
AI Document Data Extraction: Parse PDFs and Web Pages into Structured JSON with OpenAI
Download icon
Script icon
ChatCompletion icon
Queries icon
SlackIncomingWebhook icon
AIData
Run a Scheduled ETL Sync with Airbyte Cloud
Sync icon
Schedule icon
Data
See How

New to Kestra?

Use blueprints to kickstart your first workflows.

Get Started