Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting

TL;DR

Gaussian-DK tackles the challenge of multi-view brightness inconsistency in dark environments by decoupling the physical radiance field from camera imaging using anisotropic 3D Gaussians and a camera-response pipeline. The method adds a per-Gaussian light feature and a log-domain tone-mapper, along with a step-based gradient scaling strategy to suppress near-camera floaters, enabling real-time rendering with high fidelity. A new 12-scene nighttime dataset demonstrates substantial performance gains over 3DGS, NeRF-W, and HDR-NeRF, and the approach supports light-up image synthesis by adjusting exposure. Overall, the work broadens real-time neural rendering to imperfect, real-world lighting conditions and provides practical tools for nighttime view synthesis and visualization.

Abstract

3D Gaussian Splatting has recently emerged as a powerful representation that can synthesize remarkable novel views using consistent multi-view images as input. However, we notice that images captured in dark environments where the scenes are not fully illuminated can exhibit considerable brightness variations and multi-view inconsistency, which poses great challenges to 3D Gaussian Splatting and severely degrades its performance. To tackle this problem, we propose Gaussian-DK. Observing that inconsistencies are mainly caused by camera imaging, we represent a consistent radiance field of the physical world using a set of anisotropic 3D Gaussians, and design a camera response module to compensate for multi-view inconsistencies. We also introduce a step-based gradient scaling strategy to constrain Gaussians near the camera, which turn out to be floaters, from splitting and cloning. Experiments on our proposed benchmark dataset demonstrate that Gaussian-DK produces high-quality renderings without ghosting and floater artifacts and significantly outperforms existing methods. Furthermore, we can also synthesize light-up images by controlling exposure levels that clearly show details in shadow areas.

Figures (7)

• Figure 1: The muti-view brightness inconsistencies of images captured in the dark environment.
• Figure 2: The overall pipeline of Gaussian-DK. We represent a consistent radiance field of the physical world using 3D Gaussians. Each Gaussian is additionally attached with a learnable light feature. Under a specified viewpoint, we rasterize the Gaussians to obtain a 2D light feature map and a radiance map. The feature map is combined with the exposure level to produce a pixel-wise lightness map, which is further used to modulate the rasterized 2D radiance map by pixel-wise multiplication. Finally, we utilize a CNN tone-mapper to convert the modulated radiance values into pixel values, achieving correct brightness effects.
• Figure 3: The challenging "Snowman" scene with viewpoint variations and zoom-in capturing. While NeRF-based methods fail and 3DGS generates plenty of floaters, our method can still synthesize favorable images that are similar to Ground Truths.
• Figure 4: Qualitative comparisons on our benchmark dataset. Our approach can eliminate floaters and produce images with rich details.
• Figure 5: Ablation results. Artifacts appear when any of our components are removed (recommend zooming in for better visualization).