Maple: A Multi-agent System for Portable Deep Learning across Clusters
Molang Wu, Zhao Zhang
TL;DR
Maple tackles the portability problem of distributed deep learning on heterogeneous GPU clusters by embedding a four-agent pipeline (extraction, template retrieval, verification, debugging) around a library of real-world launch templates. It translates natural language job descriptions into runnable, cluster-specific launch scripts, verified via mini-apps and refined through LLM-assisted debugging and online retrieval. Across nine HPC centers and multiple DL paradigms, Maple achieves high accuracy (around 95% in reported results) with a compact (<10B parameter) pipeline, demonstrating strong portability and efficiency without relying on large external models. The work highlights a practical, scalable approach to portable distributed DL, reducing manual scripting and enabling non-expert users to run across diverse HPC environments.
Abstract
Training deep learning (DL) models across Graphics Processing Unit (GPU) clusters is technically challenging. One aspect is that users have to compose command lines to adapt to the heterogeneous launchers, schedulers, affinity options, DL framework arguments, and environment variables. Composing correct command lines is error-prone and can easily frustrate users, impeding research or wasting resources. In this work, we present Maple, a multi-agent system that generates correct DL command lines with users' natural language input. Maple consists of four agents with the functionalities of information extraction, template retrieval, command line verification, and error correction. We evaluate Maple on nine GPU clusters across national computing centers in the U.S., five representative deep learning model families, and four commonly used parallel DL training paradigms. Our experiments also cover schedulers of SLURM and PBS and heterogeneous architectures, such as NVIDIA A100/H200 GPUs and Intel Max series GPUs. Maple achieves 92.0% accuracy in generating command lines across the 567 test cases. Leverage multiple language models with an aggregated size of 10B parameters, Maple delivers comparable performance to the state-of-the-art models of GPT-5, Claude, and Gemini. Together, these results highlight Maple's practical value in enabling portable and scalable distributed DL across heterogeneous HPC environments.