HDRGS: High Dynamic Range Gaussian Splatting

TL;DR

The High Dynamic Range Gaussian Splatting (HDR-GS) method improves HDR scene recovery accuracy and integrates a novel coarse-to-fine strategy to speed up model convergence, enhancing robustness against sparse viewpoints and exposure extremes, and preventing local optima.

Abstract

Recent years have witnessed substantial advancements in the field of 3D reconstruction from 2D images, particularly following the introduction of the neural radiance field (NeRF) technique. However, reconstructing a 3D high dynamic range (HDR) radiance field, which aligns more closely with real-world conditions, from 2D multi-exposure low dynamic range (LDR) images continues to pose significant challenges. Approaches to this issue fall into two categories: grid-based and implicit-based. Implicit methods, using multi-layer perceptrons (MLP), face inefficiencies, limited solvability, and overfitting risks. Conversely, grid-based methods require significant memory and struggle with image quality and long training times. In this paper, we introduce Gaussian Splatting-a recent, high-quality, real-time 3D reconstruction technique-into this domain. We further develop the High Dynamic Range Gaussian Splatting (HDR-GS) method, designed to address the aforementioned challenges. This method enhances color dimensionality by including luminance and uses an asymmetric grid for tone-mapping, swiftly and precisely converting pixel irradiance to color. Our approach improves HDR scene recovery accuracy and integrates a novel coarse-to-fine strategy to speed up model convergence, enhancing robustness against sparse viewpoints and exposure extremes, and preventing local optima. Extensive testing confirms that our method surpasses current state-of-the-art techniques in both synthetic and real-world scenarios.

Figures (10)

• Figure 1: Illustration of HDRGS. We redefine the color of Gaussian points as radiance $L$, enabling it to meet the primary requirement for reconstructing the HDR radiance field. After splatting, pixel irradiance $E'(\mathbf{p})$ is obtained. Then, the differentiable asymmetric grid $g$ maps the exposure under exposure time $t$ to LDR pixel color. The HDR image $I_h$ can be derived by applying the function $f_H(\cdot)$ to each $E'$, with detailed explanations provided in subsequent sections.
• Figure 2: The distribution of accumulated exposure. The x-axis represents the exposure values from a given camera viewpoint, and the y-axis represents the number of pixels with the corresponding exposure values. (a) represents the original distribution of accumulated exposure, while (b) represents the distribution after applying time scaling $f_t (\cdot)$.
• Figure 3: Comparison of LDR Image Quality for Novel View Renderings. The image in the top right of each rendered image is a zoomed-in section selected by the red rectangle. The mse error heatmap for each rendered image is shown in the bottom right. "Ours*" represents our method without $\mathcal{L}_u$
• Figure 4: Comparison of HDR image quality for novel viewpoints on real scenes. The first row is GT LDR images, the second row HDR images are rendered by HDRNeRFhuang2022hdr , and the third row is ours.
• Figure 5: Comparison of HDR image quality for novel viewpoints on syn scenes. (a) represents the GT HDR images, (b) depicts the HDR images rendered by HDRNeRFhuang2022hdr , while the right side (c) shows their error maps drawn by HDR-VDP. (d) presents the HDR images rendered by our method, and (e) displays the error maps of ours.