Airpaz

Since its creation in 2011, Airpaz has embarked on an ambitious journey to transform the travel industry. Initially focusing on flight ticket sales, the company has evolved to encompass a broad offer of travel services, including international flights and accommodation bookings. As a third-party aggregator, Airpaz has successfully facilitated travel for over 3 million people, leveraging a diverse range of airlines and accommodation partners. Their commitment to enhancing the travel booking experience is reflected in their user-centric platform, which offers features like flexible payments, effortless booking management, and best in class customer support.

Operating in the Southeast Asia market, with a footprint extending to Australia, China, Japan, and beyond, Airpaz aims to cater to the global travel community. Airpaz required a robust solution to orchestrate their data workflows efficiently.

The search for an optimal data orchestration tool led them to Kestra. Unlike other platforms they considered, Kestra stood out for its simplicity, developer-friendly environment, and visual interface for flow design. The YAML-like configuration of Kestra resonated with Airpaz's preference for straightforward, manageable workflows.

Tech Stack Overview

Airpaz's technology stack includes ClickHouse for database management, another database for caching, and custom Python scripts for data ingestion. The data orchestration layer is powered by Kestra, which has become integral to automating and monitoring data workflows. Data visualization is handled, with an in-house application developed for business intelligence.

• Databases: ClickHouse,
• Data Orchestration: Kestra
• Data Collector: In house tool
• Data Visualization: In house tool for data exploration
• In-House Applications: Python (data ingestion)

Airpaz's Orchestration Challenge

Airpaz embarked on enhancing its data management system amidst complex requirements. The primary goal was to adopt a tool that integrated with Airpaz's tech stack but also accommodated the custom logic essential for their diverse data operations. The challenge was also to integrate with the variety of data sources Airpaz relied on, ranging from CRM systems to product databases and external sources like Google Ads. Orchestrating data pipelines across these varied inputs required a solution that could offer both flexibility and simplicity, enabling the creation of efficient workflows without overwhelming the development team.

Why Kestra?

Kestra addressed Airpaz's needs on multiple fronts. Its YAML-based configuration provided a simpler, more readable way to define data workflows, contrasting sharply with the more complex scripting and configuration demands of other tools. This simplicity did not come at the expense of functionality; Kestra's design allows for the incorporation of complex custom logic within workflows, catering to Airpaz's specific data processing requirements.

Moreover, Kestra's visual interface streamlined the workflow design process, making it accessible for developers and data engineers to conceptualize, implement, and manage data flows. This visual approach, combined with the YAML configurations, significantly reduced the learning curve and operational complexity.

Kestra's adaptability was another key factor in its selection. Its ability to connect with Airpaz's existing infrastructure, including ClickHouse for databases and custom Python scripts for data ingestion, ensured that Airpaz could leverage its full tech stack without needing to overhaul its systems. This integration extends to real-time applications and caching mechanisms, as well as reporting and analysis through an in-house application, creating a cohesive and efficient data ecosystem.

Outcomes with Kestra at Airpaz

Implementing Kestra transformed Airpaz's data orchestration, automating previously manual data pipelines and establishing a higher degree of control over data flows. This automation extended to comprehensive data captures from Google Ads, facilitating detailed analysis without data gaps.

Kestra's clear and manageable configuration files have also streamlined the process of maintaining and upgrading data pipelines, enhancing system reliability. The establishment of reliable data pipelines has been crucial in ensuring the availability of up-to-date data for real-time applications, and business reporting.

What's Next

The successful integration of Kestra into Airpaz's data management framework underscores their commitment to employing advanced technological solutions to improve travel experiences for customers globally.