Introduction
HTCH is creating a new kind of architectural design collaboration tool. One with Architects at the heart. Architects and designers often have to manage huge files and handle different versions, with different file formats. It takes time and storage space. To solve those challenges and provide a modern approach to architects, HTCH rely on Kestra for orchstrating their new tool.
Technology Stack
Native three-dimensions architect files are not suited for the web. HTCH's mission is to create a collaborative web platform that can expose those models in an efficient way. They need to upload files to cloud storage and connect APIs together so their engine can process 3D models and expose them to their users.
To manage this complex data landscape, HTCH implemented a multifaceted technology stack:
- Kafka: to send events between the web API where users collaborate and trigger Kestra flows to process data.
- AWS S3: to store files and data in the cloud.
- Discord: whenever a file transformation succeeds or fails, information is notified into a dedicated Discord channel.
- Node & Python: to process and optimize files using the gtlf-transform library and some home made scripts.
- Kestra: The orchestration layer that ties all the above technologies together, simplifying complex workflows and automations.
Kestra's Solution
HTCH uses Kestra to compress input models into web-friendly format and to connect their different systems together. Kestra offers two big advantages:
- Event-trigger Pattern: whenever a user uploads massive files to the web browser, it sends events through Kafka that will trigger Kestra to run flows and process data. In doing so, they keep their services and systems uncoupled from each other while Kestra manages them altogether.
- Moving fast: Kestra ease of installation allows them to move fast and have a production grade project in a matter of a few days. The versatility of Kestra to connect to S3 storage, run custom scripts and being triggered after Kafka events make the whole project easy to manage.
By adopting Kestra, HTCH has been able to create their app fast without limiting development and system integration.
Next Steps
HTCH is already very happy with its system design. Having abstracted file handling, connected to S3 file storage and trigger on new Kafka events gather all the APIs and system together. There are now looking to the following next steps to fine tune the whole platform:
- Move from Restack to AWS EC2: HTCH is currently using Restack to have Kestra deployed without any management needed. However, it has a cost, and they are willing to directly use an AWS EC2 instance with Postgres RDS for the database and S3 for internal storage to have a simple yet efficient and robust Kestra installation on the cloud.
- Use the Discord plugin: Kestra now supports Discord as a proper plugin so they can better integrate with this messaging system.
