Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CGSim: A Simulation Framework for Large Scale Distributed Computing Environment

Sairam Sri Vatsavai, Raees Khan, Kuan-Chieh Hsu, Ozgur O. Kilic, Paul Nilsson, Tatiana Korchuganova, David K. Park, Sankha Dutta, Yihui Ren, Joseph Boudreau, Tasnuva Chowdhury, Shengyu Feng, Jaehyung Kim, Scott Klasky, Tadashi Maeno, Verena Ingrid Martinez, Norbert Podhorszki, Frédéric Suter, Wei Yang, Yiming Yang, Shinjae Yoo, Alexei Klimentov, Adolfy Hoisie

TL;DR

CGSim addresses the challenge of evaluating performance and policy effects in WLCG-scale distributed computing by providing a scalable, faithful simulation framework built on SimGrid. It introduces modular policy plugins, real-time visualization dashboards, and automatic generation of event-level datasets to support ML-based performance modeling. Calibration against PanDA traces yields substantial accuracy improvements and near-linear multi-site scaling, with up to a 6x advantage for distributed workloads over single-site runs. By enabling safe, rapid experimentation of scheduling and data-management strategies on commodity hardware, CGSim accelerates the design and validation of AI-assisted performance optimizations for large-scale scientific infrastructures.

Abstract

Large-scale distributed computing infrastructures such as the Worldwide LHC Computing Grid (WLCG) require comprehensive simulation tools for evaluating performance, testing new algorithms, and optimizing resource allocation strategies. However, existing simulators suffer from limited scalability, hardwired algorithms, lack of real-time monitoring, and inability to generate datasets suitable for modern machine learning approaches. We present CGSim, a simulation framework for large-scale distributed computing environments that addresses these limitations. Built upon the validated SimGrid simulation framework, CGSim provides high-level abstractions for modeling heterogeneous grid environments while maintaining accuracy and scalability. Key features include a modular plugin mechanism for testing custom workflow scheduling and data movement policies, interactive real-time visualization dashboards, and automatic generation of event-level datasets suitable for AI-assisted performance modeling. We demonstrate CGSim's capabilities through a comprehensive evaluation using production ATLAS PanDA workloads, showing significant calibration accuracy improvements across WLCG computing sites. Scalability experiments show near-linear scaling for multi-site simulations, with distributed workloads achieving 6x better performance compared to single-site execution. The framework enables researchers to simulate WLCG-scale infrastructures with hundreds of sites and thousands of concurrent jobs within practical time budget constraints on commodity hardware.

CGSim: A Simulation Framework for Large Scale Distributed Computing Environment

TL;DR

CGSim addresses the challenge of evaluating performance and policy effects in WLCG-scale distributed computing by providing a scalable, faithful simulation framework built on SimGrid. It introduces modular policy plugins, real-time visualization dashboards, and automatic generation of event-level datasets to support ML-based performance modeling. Calibration against PanDA traces yields substantial accuracy improvements and near-linear multi-site scaling, with up to a 6x advantage for distributed workloads over single-site runs. By enabling safe, rapid experimentation of scheduling and data-management strategies on commodity hardware, CGSim accelerates the design and validation of AI-assisted performance optimizations for large-scale scientific infrastructures.

Abstract

Large-scale distributed computing infrastructures such as the Worldwide LHC Computing Grid (WLCG) require comprehensive simulation tools for evaluating performance, testing new algorithms, and optimizing resource allocation strategies. However, existing simulators suffer from limited scalability, hardwired algorithms, lack of real-time monitoring, and inability to generate datasets suitable for modern machine learning approaches. We present CGSim, a simulation framework for large-scale distributed computing environments that addresses these limitations. Built upon the validated SimGrid simulation framework, CGSim provides high-level abstractions for modeling heterogeneous grid environments while maintaining accuracy and scalability. Key features include a modular plugin mechanism for testing custom workflow scheduling and data movement policies, interactive real-time visualization dashboards, and automatic generation of event-level datasets suitable for AI-assisted performance modeling. We demonstrate CGSim's capabilities through a comprehensive evaluation using production ATLAS PanDA workloads, showing significant calibration accuracy improvements across WLCG computing sites. Scalability experiments show near-linear scaling for multi-site simulations, with distributed workloads achieving 6x better performance compared to single-site execution. The framework enables researchers to simulate WLCG-scale infrastructures with hundreds of sites and thousands of concurrent jobs within practical time budget constraints on commodity hardware.

Paper Structure

This paper contains 14 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Simulation Framework
  4. Architecture Overview
  5. Simulation Engine and Core Components
  6. Plugins
  7. Evaluation
  8. Case Study: ATLAS Computing Grid
  9. Calibration and Validation
  10. Experimental Results
  11. Scalability
  12. Event Level Simulation Snapshot
  13. Visualization and Monitoring
  14. Conclusion

Figures (5)

  • Figure 1: (a) Architecture Overview of CGSim (b) ATLAS computing Grid (c) Calibration methodology.
  • Figure 2: Abstract class to allow users to define their own allocation policies.
  • Figure 3: Job walltime calibration of CGSim for single-core and multi-core jobs across the 50 sites of WLCG. For brevity, we are plotting only 10 sites. Geometric mean is computed for all the sites.
  • Figure 4: Scalability analysis of CGSim: (a) job scaling performance with increasing workload density per single site, and (b) multi-site scaling performance with fixed job density across 1-50 computing sites.
  • Figure 5: Monitoring provides a visual analysis of the workload distribution, with hover-over details showing the jobs running on each node.