GaussHDR: High Dynamic Range Gaussian Splatting via Learning Unified 3D and 2D Local Tone Mapping

TL;DR

GaussHDR tackles HDR NVS by combining 3D Gaussian Splatting with locally adaptive tone mapping for both 3D and 2D views. A per-Gaussian context feature guides a shared local tone mapper, with a residual module enabling fine-grained adjustments, while an uncertainty model adaptively fuses the dual LDR renderings in the loss and during synthesis: $I_{merge} = \frac{U_{2d}^2 I_{3d}^* + U_{3d}^2 I_{2d}^*}{U_{3d}^2 + U_{2d}^2}$. Training proceeds in stages to stabilize the global tone mapper first and then learn local refinements, and an additional unit-exposure loss helps HDR scale on synthetic data: $\mathcal{L}_e = \| g(0) - 0.73 \|_2^2$. Across synthetic and real datasets, GaussHDR achieves state-of-the-art HDR and LDR quality by unifying 3D and 2D tone mapping with per-Gaussian context and uncertainty-guided fusion.

Abstract

High dynamic range (HDR) novel view synthesis (NVS) aims to reconstruct HDR scenes by leveraging multi-view low dynamic range (LDR) images captured at different exposure levels. Current training paradigms with 3D tone mapping often result in unstable HDR reconstruction, while training with 2D tone mapping reduces the model's capacity to fit LDR images. Additionally, the global tone mapper used in existing methods can impede the learning of both HDR and LDR representations. To address these challenges, we present GaussHDR, which unifies 3D and 2D local tone mapping through 3D Gaussian splatting. Specifically, we design a residual local tone mapper for both 3D and 2D tone mapping that accepts an additional context feature as input. We then propose combining the dual LDR rendering results from both 3D and 2D local tone mapping at the loss level. Finally, recognizing that different scenes may exhibit varying balances between the dual results, we introduce uncertainty learning and use the uncertainties for adaptive modulation. Extensive experiments demonstrate that GaussHDR significantly outperforms state-of-the-art methods in both synthetic and real-world scenarios.

Figures (11)

• Figure 1: 3D tone mapping vs. 2D tone mapping. Training with 3D tone mapping often results in inaccurate HDR rendering, while training with 2D tone mapping degrades LDR rendering quality, leading to a higher LDR RMSE metric compared to 3D tone mapping.
• Figure 2: Overview of the proposed GaussHDR. (a) We assign each 3D Gaussian with a context feature for 3D local tone mapping and uncertainty prediction. HDR irradiance, context feature, LDR color and uncertainty are simultaneously rendered onto the image plane. (b) We perform 2D local tone mapping on the rendered HDR image and feature map and predict the uncertainty. (c) We combine the dual LDR rendering results under 3D and 2D local tone mapping at the loss level and utilize their uncertainties for adaptive modulation.
• Figure 3: Residual local tone mapper and uncertainty model. We implement local tone mapper by adding a residual term to the global one. An uncertainty model is utilized to predict the uncertainty of local tone-mapping results.
• Figure 4: Qualitative LDR comparisons. Error maps in column 2 and 4 show the MSE error compared to the ground truth, where color from blue to red indicates the error from small to large. Our method can reduce LDR fitting errors in some regions.
• Figure 5: Qualitative HDR comparisons. Our method leads to stable HDR reconstruction results compared to HDR-Scaffold-GS baseline.