STRIELAD -- A Scalable Toolkit for Real-time Interactive Exploration of Large Atmospheric Datasets

TL;DR

This work tackles the challenge of visually analyzing massive, time-dependent atmospheric simulations by introducing STRIELAD, a scalable toolkit that fuses octree-based acceleration, distributed feature extraction, and level-of-detail rendering with real-time data streaming. The approach combines a preprocessing octree, a master–slave HPC-enabled backend, and a pseudo-volumetric A-Buffer rendering pipeline to support interactive sub-volume selection and shading. Key contributions include the architecture for scalable, real-time exploration on multi-terabyte datasets, demonstrated with a 2.9 TB weather dataset and integration of geospatial data sources. The framework enables practical, real-time weather/climate analysis and decision-support by enabling focused, high-fidelity visualization of large-scale simulations.

Abstract

Technological advances in high performance computing and maturing physical models allow scientists to simulate weather and climate evolutions with an increasing accuracy. While this improved accuracy allows us to explore complex dynamical interactions within such physical systems, inconceivable a few years ago, it also results in grand challenges regarding the data visualization and analytics process. We present STRIELAD, a scalable weather analytics toolkit, which allows for interactive exploration and real-time visualization of such large scale datasets. It combines parallel and distributed feature extraction using high-performance computing resources with smart level-of-detail rendering methods to assure interactivity during the complete analysis process.

Figures (13)

• Figure 1: Distributed visualization setup: The frontend determines nodes for parallel processing. The backend executes the feature extraction on a HPC cluster system and streams results back.
• Figure 2: The original dataset is partitioned into many small data blocks (left). Then, by combining eight blocks into one, an octree is generated (center). When rendering, a set of nodes (called the 'cut') is chosen so that blocks close to the virtual camera have a high resolution (right).
• Figure 3: The full dataset cannot be rendered at interactive frame rates (a). However, this is possible if a view dependent resolution is used in the distance. The error in the distance is barely noticeable.
• Figure 4: An airplane landing at Hamburg Airport. All 27000 airplanes are loaded an populate the scene with their trajectories. The length of the trajectories can be adjusted and defaults to the distance the respective airplane traveled in two minutes
• Figure 5: The sub-volumes can be colored with complex shader code. In these examples scalars are mapped on a color scale.