In-Situ High Performance Visualization for Astronomy & Cosmology
Nicola Tuccari, Eva Sciacca, Yolanda Becerra, Enric Sosa Cintero, Robert Wissing, Sijing Shen, Emiliano Tramontana
TL;DR
The paper addresses the challenge of visualizing and analyzing petascale astrophysical data without incurring prohibitive I/O overhead by proposing in-situ visualization using a distributed streaming backend. It introduces Hecuba as an Apache Cassandra Object Mapper to enable seamless online data access, and demonstrates integration with ChaNGa for $N$-body simulations and visualization platforms ParaView and VisIVO for real-time analysis. The work presents prototype implementations—a ParaView plugin and a VisIVO Importer—facilitating online data acquisition and rendering directly from the simulation pipeline, with plans to extend the ChaNGa-Hecuba integration and perform comprehensive performance studies. This approach aims to reduce data movement, enable timely insights, and adapt to evolving HPC infrastructures for Astronomy & Cosmology applications.
Abstract
The Astronomy & Cosmology (A&C) community is presently witnessing an unprecedented growth in the quality and quantity of data coming from simulations and observations. Writing results of numerical simulations to disk files has long been a bottleneck in high-performance computing. To access effectively and extract the scientific content of such large-scale data sets appropriate tools and techniques are needed. This is especially true for visualization tools, where petascale data size problems cannot be visualized without some data filtering, which reduces either the resolution or the amount of data volume managed by the visualization tool. A solution to this problem is to run the analysis and visualization concurrently (in-situ) with the simulation and bypass the storage of the full results. In particular we use Hecuba, a framework offering a highly distributed database to stream A\&C simulation data for on-line visualization. We will demonstrate the Hecuba platform integration with the Changa high performant cosmological simulator and the in-situ visualization of its N-body results with the ParaView and VisIVO tools.