Unifying Color and Lightness Correction with View-Adaptive Curve Adjustment for Robust 3D Novel View Synthesis

TL;DR

This work proposes Luminance-GS++, a 3DGS-based framework for robust NVS under diverse illumination conditions that preserves the explicit 3DGS formulation, improving reconstruction fidelity while maintaining real-time rendering efficiency.

Abstract

High-quality image acquisition in real-world environments remains challenging due to complex illumination variations and inherent limitations of camera imaging pipelines. These issues are exacerbated in multi-view capture, where differences in lighting, sensor responses, and image signal processor (ISP) configurations introduce photometric and chromatic inconsistencies that violate the assumptions of photometric consistency underlying modern 3D novel view synthesis (NVS) methods, including Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), leading to degraded reconstruction and rendering quality. We propose Luminance-GS++, a 3DGS-based framework for robust NVS under diverse illumination conditions. Our method combines a globally view-adaptive lightness adjustment with a local pixel-wise residual refinement for precise color correction. We further design unsupervised objectives that jointly enforce lightness correction and multi-view geometric and photometric consistency. Extensive experiments demonstrate state-of-the-art performance across challenging scenarios, including low-light, overexposure, and complex luminance and chromatic variations. Unlike prior approaches that modify the underlying representation, our method preserves the explicit 3DGS formulation, improving reconstruction fidelity while maintaining real-time rendering efficiency.

Figures (12)

• Figure 1: (a) An overview of Luminance-GS++, designed to address multi-view images affected by real-world illumination and color degradations via joint global and local adjustments, enabling robust 3DGS reconstruction. (b) Performance comparison between Luminance-GS++ and state-of-the-art methods cui_luminance_gscui_aleth_nerfGS-W_ECCV2024MCMC under diverse illumination and color conditions.
• Figure 2: Progression from 3DGS 3dgs to Luminance-GS cui_luminance_gs and Luminance-GS++, highlighting joint optimization of input images and 3DGS attributes through global adjustment and local refinement. Multi-view rendering examples illustrate the resulting improvements.
• Figure 3: A low-light "chair" scene from the LOM dataset cui_aleth_nerf. Our global adjustment enhances overall illumination, while our local refinement improves fine details and the quality of pseudo label $C^{out}$.
• Figure 4: An overview of the Luminance-GS++ pipeline. Up: Joint optimization of 3D Gaussians with dual color attributes $c_i$ and $c_i^{out}$ to render input images $C^{in}$ and pseudo-enhanced outputs $C^{out}$. Down: Pseudo-enhancement is achieved through global adjustment (view-adaptive color matrix mapping and curve adjustment; stages (I)--(III)) followed by local refinement using a pixel-wise residual map.
• Figure 5: An illustration of our spatial consistency loss function $\mathcal{L}_{\rm spa}$, and comparisons with the spatial consistency loss in Aleth-NeRF cui_aleth_nerf.