Integrating High Performance In-Memory Data Streaming and In-Situ Visualization in Hybrid MPI+OpenMP PIC MC Simulations Towards Exascale

TL;DR

This work tackles the I/O bottlenecks of large-scale PIC MC plasma simulations by integrating a hybrid MPI+OpenMP BIT1 with the openPMD standard and ADIOS2 SST for in-memory streaming and in-situ visualization. It implements OpenMP task-based parallelism for particle movement, enables streaming I/O via SST, and supports real-time checkpointing and visualization through openPMD, delivering measurable improvements in throughput, metadata handling, and data accessibility. Profiling across multiple HPC systems demonstrates reduced I/O overhead, scalable MPI communication, and real-time diagnostic capability, while identifying trade-offs in memory usage and read times. The approach offers a practical pathway to exascale-ready plasma simulations with faster insight and more efficient resource use, and it outlines future work to further enhance scalability and visualization integration.

Abstract

Efficient simulation of complex plasma dynamics is crucial for advancing fusion energy research. Particle-in-Cell (PIC) Monte Carlo (MC) simulations provide insights into plasma behavior, including turbulence and confinement, which are essential for optimizing fusion reactor performance. Transitioning to exascale simulations introduces significant challenges, with traditional file input/output (I/O) inefficiencies remaining a key bottleneck. This work advances BIT1, an electrostatic PIC MC code, by improving the particle mover with OpenMP task-based parallelism, integrating the openPMD streaming API, and enabling in-memory data streaming with ADIOS2's Sustainable Staging Transport (SST) engine to enhance I/O performance, computational efficiency, and system storage utilization. We employ profiling tools such as gprof, perf, IPM and Darshan, which provide insights into computation, communication, and I/O operations. We implement time-dependent data checkpointing with the openPMD API enabling seamless data movement and in-situ visualization for real-time analysis without interrupting the simulation. We demonstrate improvements in simulation runtime, data accessibility and real-time insights by comparing traditional file I/O with the ADIOS2 BP4 and SST backends. The proposed hybrid BIT1 openPMD SST enhancement introduces a new paradigm for real-time scientific discovery in plasma simulations, enabling faster insights and more efficient use of exascale computing resources.

Figures (14)

• Figure 1: Displaying the time evolution (left) and the final state (right) of the total number of particles per MPI Rank (CPU Core) for each species in BIT1, using a standard Python script that reads simulation output data, for up to 10 nodes, after the simulation has completed.
• Figure 2: A simplified diagram representing BIT1 PIC MC workflow used to simulate the plasma edge tskhakaya2004magnetisedtskhakaya2010pic. After initialization, the PIC MC algorithm cycle repeats at each time step for diagnostic output and I/O performance investigations williams2023leveraging. In orange, we highlight the I/O and plasma diagnostics step activated to integrate high-performance in-memory data streaming and in-situ visualization in BIT1 using openPMD.
• Figure 3: An Open Software Stack for Scientific I/O openPMDapiopenPMDstandardpoeschel2021transitioning integrates high-performance in-memory data streaming and in-situ visualization in BIT1 using openPMD. In orange, we highlight the BP4 engine, which is aggressively optimized for I/O efficiency at large scale by reducing metadata and aggregating data, and the SST engine, which optimizes communication and manages data transfers with minimal overhead, making it ideal for exascale systems.
• Figure 4: Hybrid BIT1 I/O and Insight Workflow using openPMD and ADIOS2 Backends.
• Figure 5: Hybrid BIT1 In-Situ Visualization Output at the End of the Simulation, as in Fig. \ref{['Streaming_and_In-Situ_Visualization']}, \ref{['Hybrid_BIT1_I/O_Workflow_using_ADIOS2_SST']} and \ref{['BIT1_openPMD_In-situ_Visualizations_Python']}.