HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting

TL;DR

A Dual Dynamic Range Gaussian point cloud model that uses spherical harmonics to fit HDR color and employs an MLP-based tone-mapper to render LDR color is designed, which can efficiently render novel HDR views and reconstruct LDR images with a user input exposure time.

Abstract

High dynamic range (HDR) novel view synthesis (NVS) aims to create photorealistic images from novel viewpoints using HDR imaging techniques. The rendered HDR images capture a wider range of brightness levels containing more details of the scene than normal low dynamic range (LDR) images. Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. In this paper, we propose a new framework, High Dynamic Range Gaussian Splatting (HDR-GS), which can efficiently render novel HDR views and reconstruct LDR images with a user input exposure time. Specifically, we design a Dual Dynamic Range (DDR) Gaussian point cloud model that uses spherical harmonics to fit HDR color and employs an MLP-based tone-mapper to render LDR color. The HDR and LDR colors are then fed into two Parallel Differentiable Rasterization (PDR) processes to reconstruct HDR and LDR views. To establish the data foundation for the research of 3D Gaussian splatting-based methods in HDR NVS, we recalibrate the camera parameters and compute the initial positions for Gaussian point clouds. Experiments demonstrate that our HDR-GS surpasses the state-of-the-art NeRF-based method by 3.84 and 1.91 dB on LDR and HDR NVS while enjoying 1000x inference speed and only requiring 6.3% training time. Code and recalibrated data will be publicly available at https://github.com/caiyuanhao1998/HDR-GS . A brief video introduction of our work is available at https://youtu.be/wtU7Kcwe7ck

Figures (6)

• Figure 1: HDR-GS vs. HDR-NeRF. Our HDR-GS achieves better PSNR in dB, SSIM, and LPIPS performance with shorter training time in minutes and faster inference speed in fps.
• Figure 2: Comparisons of point clouds (left) and rendered views (right) between the original 3DGS 3dgs (top) and our HDR-GS (bottom). (i) 3DGS 3dgs renders blurry LDR views when training with images under different exposures. Its point clouds suffer from severe color distortion and can not accurately represent the scene. In addition, 3DGS cannot control the exposure of the rendered images. (ii) Our HDR-GS can not only reconstruct clear HDR images with 3D consistency but also render LDR views with controllable exposure time $\Delta t$.
• Figure 3: Pipeline of our method. (a) SfM sfm algorithm is used to recalibrate camera parameters and initialize 3D Gaussians. (b) Dual Dynamic Range Gaussian point clouds use spherical harmonics to model the HDR color. Three MLPs are employed to tone-map the LDR color from the HDR color and user input exposure time. (c) The HDR and LDR colors are fed into two Parallel Differentiable Rasterization to render the HDR and LDR views.
• Figure 4: LDR visual comparisons on the synthetic scenes. Previous methods introduce unpleasant black spots or render blurry images. Our method controls the exposure better while reconstructing more detailed structures.
• Figure 5: LDR visual comparisons on the real scenes. Previous methods introduce unpleasant black spots or render blurry images. Our method controls the exposure better while reconstructing more detailed structures.