Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment

TL;DR

Luminance-GS addresses the sensitivity of novel view synthesis to challenging lighting by extending 3D Gaussian Splatting with per-view color matrices and a view-adaptive curve generator. It creates pseudo-enhanced images during training without changing the explicit 3DGS representation, guided by unsupervised image- and curve-level losses to enforce lighting consistency and smooth curve behavior. The approach achieves state-of-the-art quality under low-light, overexposure, and varying exposure while preserving real-time rendering speed, highlighting the practical impact for robust 3D reconstruction in diverse real-world conditions. The work paves the way for improved scene reliability in NVS systems and suggests potential extensions to scene generalization and broader camera degradation scenarios.

Abstract

Capturing high-quality photographs under diverse real-world lighting conditions is challenging, as both natural lighting (e.g., low-light) and camera exposure settings (e.g., exposure time) significantly impact image quality. This challenge becomes more pronounced in multi-view scenarios, where variations in lighting and image signal processor (ISP) settings across viewpoints introduce photometric inconsistencies. Such lighting degradations and view-dependent variations pose substantial challenges to novel view synthesis (NVS) frameworks based on Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). To address this, we introduce Luminance-GS, a novel approach to achieving high-quality novel view synthesis results under diverse challenging lighting conditions using 3DGS. By adopting per-view color matrix mapping and view-adaptive curve adjustments, Luminance-GS achieves state-of-the-art (SOTA) results across various lighting conditions -- including low-light, overexposure, and varying exposure -- while not altering the original 3DGS explicit representation. Compared to previous NeRF- and 3DGS-based baselines, Luminance-GS provides real-time rendering speed with improved reconstruction quality.

Figures (11)

• Figure 1: Up: The core idea of Luminance-GS is to perform view-adaptive adjustments for images from each viewpoint, including color matrix mapping and curve adjustment. Down: Compare with previous SOTA solutions cui_aleth_nerfGS-W_ECCV2024, our approach achieves superior performance and efficiency across different lighting conditions.
• Figure 2: Ablation analysis of different curve settings in LOM dataset cui_aleth_nerf"buu" scene: (a). All views low-light images share a single curve, (b). each view low-light image adopts an individual trainable curve and (c). our Luminance-GS solutions.
• Figure 3: Overview of Luminance-GS pipeline. Up: Our method jointly optimize 3D Gaussians with two set of color attributes $c_i$ and $c_i^{out}$ to render out input images $C^{in}$ and pseudo enhanced images $C^{out}$. Down: To translate $C^{in}$ in to view-aligned enhanced $C^{out}$, we design 3 steps: (I). per-view color matrix mapping, (II). view-adptive curve adjustment and (III). color matrix mapping back.
• Figure 4: (a). Per-view Color Matrix Mapping with learnable matrix $\mathcal{M}_k$. (b). Structure of view-adaptive curve generator, given input image $C^{in}_k$ and corresponding camera pose to predict curve bias ${\mathbb L}_k^b$. (c). Structure of view-adaptive parameters generator.
• Figure 5: Novel view synthesis results on LOM dataset cui_aleth_nerf low-light "buu", "bike" scenes and overexposure "shrub", "sofa" scenes.