HyperFLINT: Hypernetwork-based Flow Estimation and Temporal Interpolation for Scientific Ensemble Visualization

TL;DR

HyperFLINT addresses the reconstruction of flow fields and temporal interpolation in spatio-temporal scientific ensembles by explicitly conditioning on simulation parameters via a hypernetwork. The method couples a HyperNet with a streamlined FLINT* network to generate parameter-conditioned weights, enabling accurate flow estimation and high-fidelity interpolants through a fusion mask and backward warping, optimized by a loss $\,\mathcal{L} = \mathcal{L}_{rec} + \lambda_{flow} \mathcal{L}_{flow}$ with $\mathcal{L}_{rec} = \lVert D^{GT}_t - \hat{D}_{t} \rVert_{1}$ and $\mathcal{L}_{flow} = \sum_{i=1}^{N} \gamma^{N-i} \lVert F^{GT}_t - \hat{F}^{i}_{t} \rVert_{1}$, where $N=3$ and $\gamma = 0.8$. Evaluation on Nyx and Castro shows HyperFLINT outperforms baselines in both density interpolation and flow estimation, while enabling parameter-space exploration and data synthesis for configurations not explicitly simulated, all with fast inference and without extensive pretraining. This combination of parameter-aware modeling and efficient inference makes HyperFLINT a practical tool for large-scale scientific visualization and ensemble analysis.

Abstract

We present HyperFLINT (Hypernetwork-based FLow estimation and temporal INTerpolation), a novel deep learning-based approach for estimating flow fields, temporally interpolating scalar fields, and facilitating parameter space exploration in spatio-temporal scientific ensemble data. This work addresses the critical need to explicitly incorporate ensemble parameters into the learning process, as traditional methods often neglect these, limiting their ability to adapt to diverse simulation settings and provide meaningful insights into the data dynamics. HyperFLINT introduces a hypernetwork to account for simulation parameters, enabling it to generate accurate interpolations and flow fields for each timestep by dynamically adapting to varying conditions, thereby outperforming existing parameter-agnostic approaches. The architecture features modular neural blocks with convolutional and deconvolutional layers, supported by a hypernetwork that generates weights for the main network, allowing the model to better capture intricate simulation dynamics. A series of experiments demonstrates HyperFLINT's significantly improved performance in flow field estimation and temporal interpolation, as well as its potential in enabling parameter space exploration, offering valuable insights into complex scientific ensembles.

Figures (23)

• Figure 1: Overview of HyperFLINT pipeline during inference. The FLINT* deep neural network, whose weights are generated by the HyperNet, performs flow field estimation $\hat{F}_{t}$ and temporal (scalar) field interpolation $\hat{D}_{t}$, where $s < t < u$, by utilizing the available densities $D_s$ and $D_u$ from the previous and following timesteps, and their simulation parameters.
• Figure 2: HyperFLINT network architecture and pipeline during training: Given the input fields $D_s$ and $D_{u}$, and their simulation parameters, HyperFLINT predicts the $\hat{D}_{t}$ scalar field and $\hat{F}^{i}_{t}$ flow fields used in the loss function for optimizing network parameters. The HyperFLINT model consists of two key components: the HyperNet and the main network, FLINT*. The HyperNet, depicted within the red box, generates weights for the convolutional layers of FLINT* (with red outlines in the middle column of the orange box). The FLINT* model architecture and loss function are shown in the orange box. The model consists of several stacked blocks of the convolutional network, which takes $D_s$, $D_{u}$, and $t$ as input and in the $i^{th}$$\hbox{Conv Block$^{}$}$ computes estimated flows ${\hat{F}^{i}_{t\rightarrow s}}, {\hat{F}^{i}_{t\rightarrow u}}$, and fusion mask $M^i$ used for interpolation. The zoomed-in view on the right highlights the structure of a generic Conv Block. The GT density $D^{GT}_t$ and flow $F^{GT}_t$ is only used in the loss function $\mathcal{L}$. The blue dashed arrow in the right of the figure represents the gradient propagation during training, from the output of FLINT* back to the HyperNet.
• Figure 3: Illustration of the 3D backward warping $\hbox{$\mathbin{W \hbox{$\mathbin{^\leftarrow}$}}$}$: (scalar) fields $D_s$ and $D_u$ are reversely mapped according to the flow fields $\hat{F}_{t \rightarrow s}$ and $\hat{F}_{t \rightarrow u}$. The fields ${\hat{D}^{}_{t\leftarrow s}}$ and ${\hat{D}^{}_{t\leftarrow u}}$ are then reconstructed using trilinear interpolation considering the values at the coordinates shown with green dots (for the visible front surface of the cube).
• Figure 4: Nyx results
• Figure 5: Castro results