Regularized Multi-Decoder Ensemble for an Error-Aware Scene Representation Network

TL;DR

The paper tackles the lack of inference-time, coordinate-wise error metrics for SRNs in scientific visualization by introducing RMDSRN, a Regularized Multi-Decoder SRN that shares a feature-grid encoder across multiple lightweight decoders to generate a mean prediction and a coordinate-wise variance. A novel variance-regularization loss, defined via normalized variance and error densities and optimized with KL divergence, encourages the variance to track true reconstruction error, yielding RMDSRN with improved reconstruction and competitive uncertainty quality. Training employs an exponential-growth scheduler for the regularization weight, balancing accuracy and variance alignment as training progresses. Across diverse scalar-field datasets, RMDSRN demonstrates superior or near-superior data reconstruction (PSNR) with competitive variance–error correlation and enables uncertainty-aware volume rendering that enhances visualization fidelity compared to alternatives like MCD, MFVI, DE, and PV. The work offers a practical route to trustworthy SRN-based visualization by providing coordinate-level confidence estimates and integrating them into rendering workflows, potentially guiding resource allocation and data interpretation in scientific visualization pipelines.

Abstract

Feature grid Scene Representation Networks (SRNs) have been applied to scientific data as compact functional surrogates for analysis and visualization. As SRNs are black-box lossy data representations, assessing the prediction quality is critical for scientific visualization applications to ensure that scientists can trust the information being visualized. Currently, existing architectures do not support inference time reconstruction quality assessment, as coordinate-level errors cannot be evaluated in the absence of ground truth data. We propose a parameter-efficient multi-decoder SRN (MDSRN) ensemble architecture consisting of a shared feature grid with multiple lightweight multi-layer perceptron decoders. MDSRN can generate a set of plausible predictions for a given input coordinate to compute the mean as the prediction of the multi-decoder ensemble and the variance as a confidence score. The coordinate-level variance can be rendered along with the data to inform the reconstruction quality, or be integrated into uncertainty-aware volume visualization algorithms. To prevent the misalignment between the quantified variance and the prediction quality, we propose a novel variance regularization loss for ensemble learning that promotes the Regularized multi-decoder SRN (RMDSRN) to obtain a more reliable variance that correlates closely to the true model error. We comprehensively evaluate the quality of variance quantification and data reconstruction of Monte Carlo Dropout, Mean Field Variational Inference, Deep Ensemble, and Predicting Variance compared to the proposed MDSRN and RMDSRN across diverse scalar field datasets. We demonstrate that RMDSRN attains the most accurate data reconstruction and competitive variance-error correlation among uncertain SRNs under the same neural network parameter budgets.

Figures (9)

• Figure 1: RMDSRN for volumetric data representation with prediction variance quantification for confidence assessment. A feature grid SRN can be adapted to RMDSRN by training multiple decoders with the reconstruction loss introduced in \ref{['sec:esrn']} combing the weighted variance regularization loss detailed in \ref{['sec:resrn']} and \ref{['sec:scheduler']}.
• Figure 2: Illustration of the undesired overconfident variance problem of uncertain SRNs. Despite being the most inaccurate in the bottom thin structure as the error image shows, an uncertain SRN can fail to capture it with the highest variance elsewhere in the domain.
• Figure 3: Visualization of the combined variance regularization strength, which is the learning rate decayed with cosine annealing multiplied by $\lambda(t)$ from \ref{['eq:Lvar_scheduler']}, at each step of training with varying $\lambda_{max}$ for the scheduler.
• Figure 4: Volume renderings of reconstructed data for Nyx (left) and Asteroid (Right) as well as the ground truth in blue borders. Both image-level evaluation metrics and the enlarged views show the renderings from RMDSRN maintain the highest fidelity overall and reproduce feature structures the most clearly as evidenced by SSIM and the perceptual loss.
• Figure 5: Volume renderings of the highest error (top) versus variance (bottom) obtained from uncertain SRNs of Plume on the left and Supernova on the right. RMDSRN more precisely recovers fine-grained error patterns. While PV attains a high variance-error correlation as a strong competitor, the variance can be oversmooth, potentially overlooking detailed error structures.