Apple's ML team orchestrates large-scale data pipelines with Kestra

Apple's 200-engineer ML team replaced Airflow with Kestra to orchestrate large-scale ETL workloads with declarative syntax and robust fault tolerance.

Apple's 200-engineer ML team replaced Airflow with Kestra to orchestrate large-scale ETL workloads with declarative syntax and robust fault tolerance.
/_astro/logo.BdYnjE1O.svg

Industry

Technology

Headquarter

Cupertino, USA

Solution

Apple is a global technology leader whose services, including the App Store, Apple Music, and device ecosystems, generate some of the world's largest data volumes, processed daily by a 200-engineer ML team.

Data Stack

Kestra

Kestra

Commands

Upload

Query

Parallel

Share this story

I want to highlight their robustness, which is crucial at our scale. Few companies operate at this level, especially in AI/ML.

Senior Engineering Manager

Apple ML Team
See How

Experience Kestra Today

What would change if your ML data pipelines processed at massive scale, orchestrating complex ETL workloads between your data warehouse and ML platform with declarative simplicity?

Similar Kestra Stories

Displayce Optimized Workflow Orchestration and Enhanced Data Management

Displayce Optimized Workflow Orchestration and Enhanced Data Management


HTCH Building The Best Architect Collaborative Web Tool with Kestra

HTCH Building The Best Architect Collaborative Web Tool with Kestra


Reglo, Automating ETL process with a Simple Slack Command

Reglo, Automating ETL process with a Simple Slack Command