DenoiseGS: Gaussian Reconstruction Model for Burst Denoising

TL;DR

DenoiseGS introduces a fast, feed-forward burst denoising framework based on 3D Gaussian Splatting to handle large motions and heavy noise. It adds Gaussian Self-Consistency (GSC) that uses clean-input predictions as 3D guidance and a Log-Weighted Frequency (LWF) loss to preserve fine details. The method surpasses state-of-the-art NeRF-based approaches on burst denoising and noisy novel view synthesis while achieving over 250x faster inference. It also extends to novel view synthesis under noise and provides a practical plug-in for mobile photography.

Abstract

Burst denoising methods are crucial for enhancing images captured on handheld devices, but they often struggle with large motion or suffer from prohibitive computational costs. In this paper, we propose DenoiseGS, the first framework to leverage the efficiency of 3D Gaussian Splatting for burst denoising. Our approach addresses two key challenges when applying feedforward Gaussian reconsturction model to noisy inputs: the degradation of Gaussian point clouds and the loss of fine details. To this end, we propose a Gaussian self-consistency (GSC) loss, which regularizes the geometry predicted from noisy inputs with high-quality Gaussian point clouds. These point clouds are generated from clean inputs by the same model that we are training, thereby alleviating potential bias or domain gaps. Additionally, we introduce a log-weighted frequency (LWF) loss to strengthen supervision within the spectral domain, effectively preserving fine-grained details. The LWF loss adaptively weights frequency discrepancies in a logarithmic manner, emphasizing challenging high-frequency details. Extensive experiments demonstrate that DenoiseGS significantly exceeds the state-of-the-art NeRF-based methods on both burst denoising and novel view synthesis under noisy conditions, while achieving 250$\times$ faster inference speed. Code and models are released at https://github.com/yscheng04/DenoiseGS.

Figures (7)

• Figure 1: DenoiseGS $vs.$ NAN pearl2022nan. Our DenoiseGS achieves better PSNR, SSIM, LPIPS values with less FLOPs (in GFLOPs) and faster inference speed (in FPS).
• Figure 2: Noise parameters used during training and evaluation. Values (1-20) beside the point indicate gain levels defined in KPN mildenhall2018burst. $\sigma_{\max} = \sqrt{\sigma_r^2 + \sigma_s^2}$ represents the maximum noise.
• Figure 3: Pipeline. (a) Multi-view noisy inputs and camera conditions are processed by Transformer blocks to predict per-pixel Gaussian. During training, clean inputs are also fed into the model to generate high-quality Gaussian point clouds. (b) The high-quality point cloud is then used as guidance for the point cloud reconstructed from noisy inputs with our proposed Gaussian self-consistency loss. (c) The denoised result is further supervised in frequency domain with our proposed log-weighted frequency loss.
• Figure 4: From left to right: (a) clean image; (b) scene depth (c) depth map predicted by GS-LRM on clean inputs; (d) depth map predicted by GS-LRM on noisy inputs.
• Figure 5: Qualitative comparison of burst denoising results on the RE10K-N dataset at gain 8. Key regions for comparison are highlighted with colored rectangles. While competing methods suffer from residual noise, over-smoothing, and visual artifacts, our approach robustly restores fine details and sharp structures from highly noisy inputs. Please zoom in for a better view.