Uncertainty Quantification in HSI Reconstruction using Physics-Aware Diffusion Priors and Optics-Encoded Measurements

TL;DR

This work tackles the ill-posed problem of reconstructing hyperspectral images from compressed RGB measurements by casting RGB-to-HSI inversion as Bayesian inference and introducing HSDiff, a diffusion-prior framework that performs posterior diffusion sampling to yield diverse, measurement-consistent HSI estimates. It couples an unconditional diffusion model trained on metamer-enriched data with a perturbed likelihood to enable uncertainty quantification under arbitrary forward operators, including optics-based encodings and CASSI. A novel metameric augmentation strategy—texture-guided black metamers and PU-basis metamers—expands spectral diversity and improves posterior calibration. Across datasets and forward models, optics-aware encoding provides more informative and better-calibrated uncertainty, underscoring the importance of forward-model design for uncertainty-aware HSI reconstruction and guiding future spectral-encoding strategies for snapshot hyperspectral imaging.

Abstract

Hyperspectral image reconstruction from a compressed measurement is a highly ill-posed inverse problem. Current data-driven methods suffer from hallucination due to the lack of spectral diversity in existing hyperspectral image datasets, particularly when they are evaluated for the metamerism phenomenon. In this work, we formulate hyperspectral image (HSI) reconstruction as a Bayesian inference problem and propose a framework, HSDiff, that utilizes an unconditionally trained, pixel-level diffusion prior and posterior diffusion sampling to generate diverse HSI samples consistent with the measurements of various hyperspectral image formation models. We propose an enhanced metameric augmentation technique using region-based metameric black and partition-of-union spectral upsampling to expand training with physically valid metameric spectra, strengthening the prior diversity and improving uncertainty calibration. We utilize HSDiff to investigate how the studied forward models shape the posterior distribution and demonstrate that guiding with effective spectral encoding provides calibrated informative uncertainty compared to non-encoded models. Through the lens of the Bayesian framework, HSDiff offers a complete, high-performance method for uncertainty-aware HSI reconstruction. Our results also reiterate the significance of effective spectral encoding in snapshot hyperspectral imaging.

Figures (15)

• Figure 1: (a) RGB image with two marked pixel locations. (b) Predicted spectra at the marked pixels from MST++ trained (NLL-trained) without aberrations. The metamer prediction (orange solid) is completely wrong from the truth (red dotted). Moreover, the true metamer spectrum falls outside the confidence interval for the prediction, indicating that the prediction is "confidently wrong". (c) Predicted spectra at the same pixels from MST++ (NLL-trained) with optics-induced aberrations. Both original and metamer spectra are predicted correctly with reasonable uncertainty (mean $\pm \ 2 \ \sigma$). In both plots, the green line shows the standard deterministic MST++ prediction as a reference.
• Figure 2: HSDiff results on CAVE. Top: original and metamer HSI cubes, averaged over the spectral dimension. Each subsequent row corresponds to the operator used in guided diffusion—None (no aberration), Grating PSF, Gaussian PSF, Rotational-diffraction PSF, and CASSI. Columns show: (i) posterior mean from 20 samples, (ii) posterior standard deviation, (iii) absolute error $|\text{mean} - \text{original}|$, and (iv) Binary Coverage Map (BCM), where black indicates ground truth outside the $95\%$ prediction interval.
• Figure 3: Top: RGB images under no aberration, Gaussian PSF, Grating PSF, Rotational PSF, and the CASSI measurement. Middle and bottom: spectral profiles at the marked pixels. Each plot shows the original spectrum, one sample black metamer, and one PU-basis metamer, along with the posterior mean $\pm \ 2\sigma$ after guided diffusion. The metamer-enriched prior broadens the credible bands—posterior samples typically lie between the ground truth and the metameric alternatives. In contrast, CASSI remains close to the ground truth, since its measurement is not affected by RGB-targeted metamerism.
• Figure 4: Calibration cross-plots (mean std vs MAE). Left: CAVE. Right: ICVL. Each point is one image–operator pair (None, Grating, Gaussian, Rotational, CASSI). A clear positive trend indicates informative uncertainty.
• Figure 5: Per-image metrics across operators. Violin plots for CAVE (left) and ICVL (right): PSNR (top, higher is better), mean posterior std (middle, uncertainty proxy), and SAM (bottom, lower is better). CASSI trends higher PSNR/lower SAM overall.