Optimisation of ATLAS computing resource usage through a modern HEP Benchmark Suite via HammerCloud and Big PanDA
Natalia Szczepanek, Domenico Giordano, Ivan Glushkov, Gonzalo Menendez Borge, Alessandro Di Girolamo, Alexander Lory, Ilija Vukotic
TL;DR
This work tackles the challenge of validating declared ATLAS corepower against actual runtime performance during the HS23 transition. It implements an automated benchmarking pipeline using HammerCloud and PanDA with the HEP Benchmark Suite to collect performance and system metrics across 136 sites, storing results in OpenSearch for visualization and analysis. The analysis reveals substantial discrepancies between runtime and declared corepower, largely due to outdated declared values and partly influenced by server load; the framework enables rapid identification and remediation, improving resource accounting and operational efficiency for ATLAS and WLCG. Overall, the study provides a robust, production-oriented method to verify and refine resource declarations, promoting transparency and better decision-making in distributed computing resources.
Abstract
In April 2023, HEPScore23, the new benchmark based on HEP specific applications, was adopted by WLCG, replacing HEP-SPEC06. As part of the transition to the new benchmark, the CPU corepower published by the sites needed to be compared with the effective power observed while running ATLAS workloads. One aim was to verify the conversion rate between the scores of the old and the new benchmark. The other objective was to understand how the HEPScore performs when run on multi-core job slots, so exactly like the computing sites are being used in the production environment. Our study leverages the HammerCloud infrastructure and the PanDA Workload Management System to collect a large benchmark statistic across 136 computing sites using an enhanced HEP Benchmark Suite. It allows us to collect not only performance metrics, but, thanks to plugins, it also collects information such as machine load, memory usage and other user-defined metrics during the execution and stores it in an OpenSearch database. These extensive tests allow for an in-depth analysis of the actual, versus declared computing capabilities of these sites. The results provide valuable insights into the real-world performance of computing resources pledged to ATLAS, identifying areas for improvement while spotlighting sites that underperform or exceed expectations. Moreover, this helps to ensure efficient operational practices across sites. The collected metrics allowed us to detect and fix configuration issues and therefore improve the experienced performance.