Beyond Darkness: Thermal-Supervised 3D Gaussian Splatting for Low-Light Novel View Synthesis

TL;DR

This work tackles robust novel-view synthesis under extreme low-light by leveraging an illumination-invariant thermal modality. It introduces DTGS, a joint framework that couples Retinex-style RGB enhancement with end-to-end 3D Gaussian Splatting, guided by a thermal supervision branch, and uses a cyclic ground-truth update to break the enhancement-geometry deadlock. The Retinex decomposition $I_{ ext{low}} = R \odot L$ with $I_{ ext{enh}} = R \odot L'$ is embedded in the 3DGS loop, enabling physically interpretable reflectance-illumination separation and thermally guided reconstruction. The approach is validated on the new RGBT-LOW dataset, where DTGS achieves superior radiometric consistency, geometric fidelity, and color stability compared with enhancement-plus-3D baselines, demonstrating strong practical potential for illumination-aware 3D reconstruction in challenging environments.

Abstract

Under extremely low-light conditions, novel view synthesis (NVS) faces severe degradation in terms of geometry, color consistency, and radiometric stability. Standard 3D Gaussian Splatting (3DGS) pipelines fail when applied directly to underexposed inputs, as independent enhancement across views causes illumination inconsistencies and geometric distortion. To address this, we present DTGS, a unified framework that tightly couples Retinex-inspired illumination decomposition with thermal-guided 3D Gaussian Splatting for illumination-invariant reconstruction. Unlike prior approaches that treat enhancement as a pre-processing step, DTGS performs joint optimization across enhancement, geometry, and thermal supervision through a cyclic enhancement-reconstruction mechanism. A thermal supervisory branch stabilizes both color restoration and geometry learning by dynamically balancing enhancement, structural, and thermal losses. Moreover, a Retinex-based decomposition module embedded within the 3DGS loop provides physically interpretable reflectance-illumination separation, ensuring consistent color and texture across viewpoints. To evaluate our method, we construct RGBT-LOW, a new multi-view low-light thermal dataset capturing severe illumination degradation. Extensive experiments show that DTGS significantly outperforms existing low-light enhancement and 3D reconstruction baselines, achieving superior radiometric consistency, geometric fidelity, and color stability under extreme illumination.

Figures (4)

• Figure 1: RGB-only enhancement struggles to maintain structural coherence and color consistency in extremely low-light scenes. Our method delivers more consistent colors and superior contrast.
• Figure 2: Overview of the DTGS Framework. Given multi-view low-light and thermal images of a scene, our method performs joint enhancement and 3D reconstruction through cyclic optimization. The thermal modality guides Retinex-based enhancement of low-light images, producing $I_{\text{enh}}$ that progressively updates the ground truth $GT^{(t)} = (1-\alpha^{(t)}) \cdot GT^{(t-1)} + \alpha^{(t)} \cdot I_{\text{enh}}^{(t)}$ for 3D Gaussian Splatting training. The total loss $\mathcal{L}_{\text{total}} = \lambda_{\text{enh}} \cdot \mathcal{L}_{\text{enh}} + \lambda_{\text{gs}} \cdot \mathcal{L}_{\text{gs}} + \lambda_{\text{therm}} \cdot \mathcal{L}_{\text{therm}}$ is optimized with adaptive four-stage weight scheduling. Thermal consistency loss ensures cross-modal coherence, enabling bidirectional gradient flow between enhancement and reconstruction modules.
• Figure 3: Qualitative comparison on the RGBT-LOW dataset. Our method produces more consistent color restoration and structural fidelity across views. Compared with other enhancement-based approaches, DTGS effectively preserves object integrity and avoids enhancement tearing, geometric distortion, or color shifts commonly seen in partially enhanced regions.include gaussian-DK ye2024gaussian,3dgs,retinexformer, RaDe-GS zhang2024rade,Thermal Gaussian
• Figure 4: Qualitative comparison of ablation variants.