Phantora: Maximizing Code Reuse in Simulation-based Machine Learning System Performance Estimation

TL;DR

Phantora introduces a hybrid GPU cluster simulator that reuses unmodified ML framework code by running them inside containers and intercepting GPU and network operations for performance estimation. It integrates an event-driven network simulator with a time-rollback mechanism and CPU-memory sharing to enable accurate, scalable simulation using a single GPU per container. Evaluations across Megatron, DeepSpeed, and Torch Titan show Phantora achieves accuracy comparable to static workload simulators while dramatically reducing setup and maintenance burden, and a case study demonstrates its usefulness for activation recomputation analysis. The approach offers a practical, reusable path for rapid ML-system performance estimation on modest hardware, with open-source availability for broader adoption.

Abstract

Modern machine learning (ML) training workloads place substantial demands on both computational and communication resources. Consequently, accurate performance estimation has become increasingly critical for guiding system design decisions, such as the selection of parallelization strategies, cluster configurations, and hardware provisioning. Existing simulation-based performance estimation requires reimplementing the ML framework in a simulator, which demands significant manual effort and is hard to maintain as ML frameworks evolve rapidly. This paper introduces Phantora, a hybrid GPU cluster simulator designed for performance estimation of ML training workloads. Phantora executes unmodified ML frameworks as is within a distributed, containerized environment. Each container emulates the behavior of a GPU server in a large-scale cluster, while Phantora intercepts and simulates GPU- and communication-related operations to provide high-fidelity performance estimation. We call this approach hybrid simulation of ML systems, in contrast to traditional methods that simulate static workloads. The primary advantage of hybrid simulation is that it allows direct reuse of ML framework source code in simulation, avoiding the need for reimplementation. Our evaluation shows that Phantora provides accuracy comparable to static workload simulation while supporting three state-of-the-art LLM training frameworks out-of-the-box. In addition, Phantora operates on a single GPU, eliminating the need for the resource-intensive trace collection and workload extraction steps required by traditional trace-based simulators. Phantora is open-sourced at https://github.com/QDelta/Phantora.

Figures (14)

• Figure 1: Comparison of simulators and problems of static workload simulation. Light green boxes show unmodified components; gree boxes show minimally modified components; blue boxes show simulator components and pink boxes show user input. Trace-based simulation requires workload extraction (reversing framework logic) and costly trace collection on large clusters. SimAI relies on mocked ML frameworks.
• Figure 2: Scheduling logic needs to be reimplemented in current simulators for converting high level workload to detailed computation and communication events.
• Figure 3: Phantora architecture. Components in light green are unmodified, components in green are minimally modified, and components in blue are constructed by Phantora.
• Figure 4: An example workflow of Phantora with two ranks. ML system places computation and communications on different CUDA streams for flexibility, and use CUDA events to manage the synchronization between them. Phantora needs to correctly handle these synchronizations to achieve accurate simulation.
• Figure 5: Challenges of synchronizing time between real execution and event-driven simulation.