Mono4DGS-HDR: High Dynamic Range 4D Gaussian Splatting from Alternating-exposure Monocular Videos

TL;DR

Mono4DGS-HDR addresses 4D HDR reconstruction from unposed monocular videos with alternating exposures by proposing a two-stage Gaussian Splatting framework. The first stage learns HDR video Gaussians in an orthographic space to remove pose dependence, while the second stage transforms these to world space and refines Gaussians alongside camera poses, aided by a temporal luminance regularization for HDR stability. A new HDR video benchmark is introduced, and comprehensive experiments show state-of-the-art rendering quality, temporal coherence, and speed compared to adapted baselines. The approach advances practical HDR view synthesis from easily collected monocular footage and provides reproducible results with public priors and datasets.

Abstract

We introduce Mono4DGS-HDR, the first system for reconstructing renderable 4D high dynamic range (HDR) scenes from unposed monocular low dynamic range (LDR) videos captured with alternating exposures. To tackle such a challenging problem, we present a unified framework with two-stage optimization approach based on Gaussian Splatting. The first stage learns a video HDR Gaussian representation in orthographic camera coordinate space, eliminating the need for camera poses and enabling robust initial HDR video reconstruction. The second stage transforms video Gaussians into world space and jointly refines the world Gaussians with camera poses. Furthermore, we propose a temporal luminance regularization strategy to enhance the temporal consistency of the HDR appearance. Since our task has not been studied before, we construct a new evaluation benchmark using publicly available datasets for HDR video reconstruction. Extensive experiments demonstrate that Mono4DGS-HDR significantly outperforms alternative solutions adapted from state-of-the-art methods in both rendering quality and speed.

Figures (10)

• Figure 1: (a) Our Mono4DGS-HDR can reconstruct high-quality 4D HDR scenes from unposed monocular LDR videos with alternating exposures. (b) Compared to simply extending SplineGS splinegs, MoSca mosca and GFlow gflow to HDR mode, our approach achieves significantly better reconstruction quality.
• Figure 2: Overview of Mono4DGS-HDR. (a) We infer vision foundation models on the input alternating-exposure video to extract 2D priors, which provide scene initialization and regularization. (b) We propose a novel two-stage Gaussian optimization procedure, which includes video Gaussian training in the first stage, world Gaussian fine-tuning in the second stage, and a video-to-world Gaussian transformation strategy. The HDR Gaussians are optimized through 2D prior supervision, Gaussian motion regularization, temporal luminance regularization and HDR photometric reprojection loss.
• Figure 3: (a) Our video-world Gaussian Transformation Strategy, including dynamic/static identification, attribute transformation and re-fitting. (b) Example of transformed dynamic/static world Gaussians. (c) Without occlusion handling, the dynamic/static separation is inaccurate. (d) Without 2D covariance invariance (directly inherit scaling), the world Gaussians have unreasonable scales.
• Figure 4: Temporal luminance regularization for temporally consistent HDR appearance.
• Figure 5: HDR visual comparisons on train/test frames. Our method achieves superior quality.