Diffusion-Denoised Hyperspectral Gaussian Splatting

TL;DR

This work tackles hyperspectral 3D reconstruction for agricultural scenes, addressing limited spectral fidelity and slow rendering in NeRF-based approaches. It introduces DD-HGS, a framework that combines 3D Gaussian Splatting with a wavelength-aware SH representation, a KL-divergence–based spectral loss, and a conditional diffusion denoiser to produce explicit multi-channel hyperspectral reconstructions and novel view synthesis. The method achieves state-of-the-art performance on real hyperspectral datasets, demonstrating strong spatial detail, spectral fidelity, and efficient rendering. By enabling accurate spectral rendering across hundreds of bands, DD-HGS advances digital twins for precision agriculture and robust material analysis across spectral ranges.

Abstract

Hyperspectral imaging (HSI) has been widely used in agricultural applications for non-destructive estimation of plant nutrient composition and precise quantification of sample nutritional elements. Recently, 3D reconstruction methods, such as Neural Radiance Field (NeRF), have been used to create implicit neural representations of HSI scenes. This capability enables the rendering of hyperspectral channel compositions at every spatial location, thereby helping localize the target object's nutrient composition both spatially and spectrally. However, it faces limitations in training time and rendering speed. In this paper, we propose Diffusion-Denoised Hyperspectral Gaussian Splatting (DD-HGS), which enhances the state-of-the-art 3D Gaussian Splatting (3DGS) method with wavelength-aware spherical harmonics, a Kullback-Leibler divergence-based spectral loss, and a diffusion-based denoiser to enable 3D explicit reconstruction of the hyperspectral scenes for the entire spectral range. We present extensive evaluations on diverse real-world hyperspectral scenes from the Hyper-NeRF dataset to show the effectiveness of our DD-HGS. The results demonstrate that DD-HGS achieves the new state-of-the-art performance compared to all the previously published methods. Project page: https://dragonpg2000.github.io/DDHGS-website/

Figures (5)

• Figure 1: We propose Diffusion-Denoised Hyperspectral Gaussian Splatting (DD-HGS) for reconstructing agricultural scenes and enabling novel view synthesis under hyperspectral imaging. Compared with NeRF mildenhall2020nerf, Hyper-NeRF chen2024hyperspectralnerf, and 3DGS kerbl2023gaussians, ours can render high-quality images with fine-grained spectral details, and significantly reduce reconstruction errors.
• Figure 2: Overview of our DD-HGS framework. DD-HGS extends 3DGS with a wavelength encoder that maps positional embeddings of wavelength through an MLP to learn wavelength-dependent SH offsets, a spectral loss aligning predicted and ground truth spectral distributions, and a conditional diffusion module that refines the noisy 3DGS rendering to improve its spectral and spatial fidelity.
• Figure 3: Qualitative comparisons on BaySpec and Surface Optics datasets (750–768 nm). Each column shows renderings and difference heatmaps for NeRF, Hyper-NeRF, 3DGS, and DD-HGS. Our method preserves fine structural details and spectral fidelity.
• Figure 4: Qualitative results on the Anacampseros scene across three different wavelength ranges: 400nm to 418nm, 750nm to 768nm, 1082nm to 1100nm. The rendered images and difference heatmaps against the ground truth demonstrate the spectral fidelity and spatial consistency of the reconstruction results, particularly under challenging near-infrared and ultraviolet conditions. In addition, we visualize the reconstructed pixel intensities across all the spectral channels of three randomly selected points in the rightmost column. Compared to the baselines, our method exhibits the highest similarity to the ground truth.
• Figure 5: Qualitative ablation study on the Caladium scene. Rendered images and difference heatmaps w.r.t. ground truth are shown. The wavelength encoder (WE) and spectral loss (SL) progressively reduce detail artifacts and spectral distortions, leading to higher spatial and spectral reconstruction accuracy.