EuroHPC SPACE CoE: Redesigning Scalable Parallel Astrophysical Codes for Exascale

TL;DR

The paper discusses the SPACE CoE initiative to address exascale challenges in astrophysical simulations by re-engineering flagship codes (Pluto, OpenGadget3, Ramses, iPIC3D, ChaNGa, BHAC, FIL/GRACE) through co-design, profiling, and GPU-accelerated optimizations. It details advances in modular multiphysics frameworks, AMR and GPU porting, and scalable parallelism across CPU and GPU platforms, complemented by energy-efficiency analyses and targeted co-design with HW providers. It also covers extreme data processing and visualization strategies, including in-situ analysis, cinematic visualisation workflows, and three ML-informed applications to enhance interpretation, inference, and radiative transfer approximations within large simulations. Additionally, SPACE provides community-oriented services, standardization efforts for I/O and metadata (FAIR/IVOA), and extensive training activities to accelerate adoption and collaboration across European HPC and astrophysics communities. Collectively, the work demonstrates tangible improvements in scalability, portability, and energy efficiency, enabling robust exascale A&C research and fostering an open, interoperable ecosystem for future exascale computing.

Abstract

High Performance Computing (HPC) based simulations are crucial in Astrophysics and Cosmology (A&C), helping scientists investigate and understand complex astrophysical phenomena. Taking advantage of exascale computing capabilities is essential for these efforts. However, the unprecedented architectural complexity of exascale systems impacts legacy codes. The SPACE Centre of Excellence (CoE) aims to re-engineer key astrophysical codes to tackle new computational challenges by adopting innovative programming paradigms and software (SW) solutions. SPACE brings together scientists, code developers, HPC experts, hardware (HW) manufacturers, and SW developers. This collaboration enhances exascale A&C applications, promoting the use of exascale and post-exascale computing capabilities. Additionally, SPACE addresses high-performance data analysis for the massive data outputs from exascale simulations and modern observations, using machine learning (ML) and visualisation tools. The project facilitates application deployment across platforms by focusing on code repositories and data sharing, integrating European astrophysical communities around exascale computing with standardised SW and data protocols.

Figures (7)

• Figure 1: gPluto weak scaling results for the 3D Orszag-Tang on MareNostrum 5 ACC, MeluXina GPU nodes, and Leonardo (Booster and DCGP) (left). Snapshots of the magnetic energy density ($\frac{1}{2}B^2$) at time t = 32.5 (right). Here, time is expressed in units of the light-crossing time of the sheet length.
• Figure 2: OpenGadget3 strong scaling results of different gravity-only boxes on the Leonardo Booster using three test cases (left). Visualisation of Box3 from the Magneticum Pathfinder simulation set (right). The shown region spans a total size of $\sim180$ Mpc and contains $\sim3.5*10^{8}$ dark matter, gas, star, and black hole particles (image credit: B. Seidel).
• Figure 3: Weak scaling of Ramses (before optimisation) on the CPU partition of several EuroHPC systems for the Sedov test case with 128$^3$ cells per core (left). Projected image of the cosmological volume test case (right).
• Figure 4: Current-density distribution of a high-resolution 3D simulation of relativistic magnetic reconnection with iPIC3D (left). Weak scaling of iPIC3D on LUMI-C (right).
• Figure 5: Magnetic field strength distribution from a high-resolution galaxy simulation performed with ChaNGa (left). Weak scaling performance of ChaNGa across multiple EuroHPC supercomputers (right).