Couler: Unified Machine Learning Workflow Optimization in Cloud
Xiaoda Wang, Yuan Tang, Tengda Guo, Bo Sang, Jingji Wu, Jian Sha, Ke Zhang, Jiang Qian, Mingjie Tang
TL;DR
Couler presents a unified cloud-based framework for ML workflow optimization that translates natural language descriptions into executable workflows across multiple engines via a unified programming interface. It integrates LLM-driven NL-to-code generation, an engine-agnostic intermediate representation, and three core optimizations: automatic artifact caching, big-workflow auto-parallelism, and automatic hyperparameter tuning. Real-world deployment at Ant Group demonstrates substantial efficiency gains in CPU/memory utilization and workflow completion rates, validating practical impact for large-scale ML pipelines. The approach reduces the burden of mastering diverse workflow APIs, enhances fault tolerance, and enables scalable AutoML across heterogeneous cloud environments. Overall, Couler advances practical ML workflow orchestration by combining NL usability with rigorous optimization and cross-engine portability.
Abstract
Machine Learning (ML) has become ubiquitous, fueling data-driven applications across various organizations. Contrary to the traditional perception of ML in research, ML workflows can be complex, resource-intensive, and time-consuming. Expanding an ML workflow to encompass a wider range of data infrastructure and data types may lead to larger workloads and increased deployment costs. Currently, numerous workflow engines are available (with over ten being widely recognized). This variety poses a challenge for end-users in terms of mastering different engine APIs. While efforts have primarily focused on optimizing ML Operations (MLOps) for a specific workflow engine, current methods largely overlook workflow optimization across different engines. In this work, we design and implement Couler, a system designed for unified ML workflow optimization in the cloud. Our main insight lies in the ability to generate an ML workflow using natural language (NL) descriptions. We integrate Large Language Models (LLMs) into workflow generation, and provide a unified programming interface for various workflow engines. This approach alleviates the need to understand various workflow engines' APIs. Moreover, Couler enhances workflow computation efficiency by introducing automated caching at multiple stages, enabling large workflow auto-parallelization and automatic hyperparameters tuning. These enhancements minimize redundant computational costs and improve fault tolerance during deep learning workflow training. Couler is extensively deployed in real-world production scenarios at Ant Group, handling approximately 22k workflows daily, and has successfully improved the CPU/Memory utilization by more than 15% and the workflow completion rate by around 17%.