Fast High Dynamic Range Radiance Fields for Dynamic Scenes

TL;DR

HDR-HexPlane presents a dynamic HDR radiance-field framework that learns from multi-exposure 2D imagery to synthesize HDR and LDR novel views of dynamic scenes. It decouples dynamic scene representation (via HexPlane) from exposure learning, using a fixed monotonic camera response function to stabilize optimization and an exposure MLP to align exposures across views. The method achieves state-of-the-art results on synthetic HDR dynamic scenes, with substantial training-time improvements over prior dynamic HDR methods. A dedicated HDR dynamic-scene dataset accompanies the work, facilitating evaluation of HDR dynamic view synthesis under varying illumination and motion. This work broadens the applicability of HDR NeRFs to real-world, time-varying scenes with nonuniform lighting, enabling more robust rendering and tone-mapped outputs.

Abstract

Neural Radiances Fields (NeRF) and their extensions have shown great success in representing 3D scenes and synthesizing novel-view images. However, most NeRF methods take in low-dynamic-range (LDR) images, which may lose details, especially with nonuniform illumination. Some previous NeRF methods attempt to introduce high-dynamic-range (HDR) techniques but mainly target static scenes. To extend HDR NeRF methods to wider applications, we propose a dynamic HDR NeRF framework, named HDR-HexPlane, which can learn 3D scenes from dynamic 2D images captured with various exposures. A learnable exposure mapping function is constructed to obtain adaptive exposure values for each image. Based on the monotonically increasing prior, a camera response function is designed for stable learning. With the proposed model, high-quality novel-view images at any time point can be rendered with any desired exposure. We further construct a dataset containing multiple dynamic scenes captured with diverse exposures for evaluation. All the datasets and code are available at \url{https://guanjunwu.github.io/HDR-HexPlane/}.

Figures (9)

• Figure 1: Our model is capable of synthesizing novel viewpoint images in dynamic scenes by capturing images with different exposure values at different time points. Additionally, it can seamlessly combine images with varying exposures and produce high dynamic range (HDR) images. With the tone mapping function applied, a better color balance is achieved, enhancing the overall visual quality of images.
• Figure 2: The overall framework of our proposed method. Firstly, we cast multiple rays from the camera and sample a series of points $x$ from each ray. These points, along with the current timestamp $t$ and direction $d$, are fed into the HexPlane module. HexPlane calculates the radiance value $E^{\prime}$ and density $\sigma$ corresponding to each point, which allows us to render HDR images using the volume rendering equation. Simultaneously, the exposure mapping module learns the corresponding logarithmic exposure coefficients $e_j$ for each image index $j$. After multiplying $E^{\prime}$ with the calculated color $c$ using the camera response function, we render the corresponding LDR image using the volume rendering equation.
• Figure 3: Results of the synthesis datasets with all the images rendered in LDR (Low Dynamic Range). Our method can render dynamic LDR and HDR (High Dynamic Range) images, whereas other methods encountered challenges. The last column shows our tone-mapped HDR image, which balances both overexposed and underexposed areas.
• Figure 4: Ablation study of replacing the Sigmoid function by CRF MLP $\phi_c$ like hdrnerf. The first column stands for ground-truth HDR images, the second column stands for replacing CRF $f$ with $\phi_c$ and added by zero-point constraint $\mathcal{L}_u$. the third raw reveals adopting $\phi_c$ only and the last row is our rendered HDR images.
• Figure 5: The comparison of Learned logarithmic exposure histogram. In the histogram, the horizontal axis represents the logarithmic values of the learned exposure coefficient $e^\prime_j$, and the vertical axis represents the number of images. The first row shows our method without the exposure MLP $\phi_e$, while the second row shows our learned exposure values, which converge to some constant value.