ATLAHS: An Application-centric Network Simulator Toolchain for AI, HPC, and Distributed Storage

TL;DR

ATLAHS addresses the need for realistic, multi-domain network performance evaluation by delivering an open-source, application-centric toolchain that traces real AI, HPC, and storage workloads and simulates them with GOAL. It extends the LogGOPSim lineage by providing a unified GOAL-based workload format, multiple backends, and support for multi-job and multi-tenant scenarios, accompanied by a public trace dataset to enable reproducibility. The authors validate ATLAHS across AI and HPC workloads, achieving consistently high accuracy and outperforming AstraSim in both runtime efficiency and trace size. Through case studies on congestion control in distributed storage and job placement in clusters, ATLAHS demonstrates practical utility for architecture and protocol design. The work offers a practical framework for robust, realistic network performance evaluation in large-scale systems and invites broader adoption through open traces and modular backends.

Abstract

Network simulators play a crucial role in evaluating the performance of large-scale systems. However, existing simulators rely heavily on synthetic microbenchmarks or narrowly focus on specific domains, limiting their ability to provide comprehensive performance insights. In this work, we introduce ATLAHS, a flexible, extensible, and open-source toolchain designed to trace real-world applications and accurately simulate their workloads. ATLAHS leverages the GOAL format to model communication and computation patterns in AI, HPC, and distributed storage applications. It supports multiple network simulation backends and handles multi-job and multi-tenant scenarios. Through extensive validation, we demonstrate that ATLAHS achieves high accuracy in simulating realistic workloads (consistently less than 5% error), while significantly outperforming AstraSim, the current state-of-the-art AI systems simulator, in terms of simulation runtime and trace size efficiency. We further illustrate ATLAHS's utility via detailed case studies, highlighting the impact of congestion control algorithms on the performance of distributed storage systems, as well as the influence of job-placement strategies on application runtimes.

Figures (13)

• Figure 1: illustrates a space-time diagram of a realistic training scenario for Large Language Models (LLMs), showing overlapping communication from data parallelism (DP) and pipeline parallelism (PP). depicts a network-level view demonstrating how PP victim flows become congested due to simultaneous DP ring allreduce communications within a two-level fat tree topology. compares the performance of Swift and MPRDMA congestion control algorithms using two synthetic microbenchmarks and the LLM training workload. Percentages indicate the performance improvement (green) or degradation (red) of Swift relative to MPRDMA.
• Figure 2: Overview of the ATLAHS toolchain. Application and hardware components are represented in shades of red. Trace generation and GOAL format processing are shown in green. Simulation is depicted in blue. For consistency, these color schemes will be used throughout the paper in all figures and diagrams.
• Figure 3: shows an example GOAL schedule of node 0 in its textual format, while shows the visualization of the same schedule as a DAG. Vertices in green are assigned to compute stream 0 to execute while vertex $l3$ in teal is assigned to be executed on compute stream 1.
• Figure 4: Example of an AI application with 2 nodes and 4 GPUs connected in a ring, where each GPU communicates with its designated receiver as indicated by the arrows. NCCL utilizes a single streaming multiprocessor (SM) to handle communication operations. Using the NCCL Simple protocol, when GPU 0 broadcasts 2 MB of data as the root, the data is divided into 4 chunks, and transmitted sequentially.
• Figure 5: An example showing the 4 stages GOAL file generation for large-scale distributed AI applications.