PanDORA: Casual HDR Radiance Acquisition for Indoor Scenes

TL;DR

PanDORA tackles the challenge of capturing true HDR radiance for indoor scenes by pairing two synchronized 360° cameras to record well-exposed and fast-exposed panoramas, enabling near-complete HDR radiance reconstruction with a two-stage NeRF pipeline. The method learns separate HDR radiance fields for each exposure and merges them through a carefully calibrated HDR fusion, aided by a fine alignment step to mitigate misregistration. The authors introduce a practical dataset of 14 real indoor scenes with ground-truth HDR panoramas and demonstrate that PanDORA outperforms contemporary HDR-NeRF baselines in both HDR quality and physically plausible relighting. This approach democratizes HDR radiance capture for NeRF-style rendering, offering a scalable, portable solution suitable for virtual production, AR/VR, and object relighting tasks.

Abstract

Most novel view synthesis methods-including Neural Radiance Fields (NeRF)-struggle to capture the true high dynamic range (HDR) radiance of scenes. This is primarily due to their dependence on low dynamic range (LDR) images from conventional cameras. Exposure bracketing techniques aim to address this challenge, but they introduce a considerable time burden during the acquisition process. In this work, we introduce PanDORA: PANoramic Dual-Observer Radiance Acquisition, a system designed for the casual, high quality HDR capture of indoor environments. Our approach uses two 360° cameras mounted on a portable monopod to simultaneously record two panoramic 360° videos: one with standard exposure and another at fast shutter speed. The resulting video data is processed by a proposed two-stage NeRF-based algorithm, including an algorithm for the fine alignment of the fast- and well-exposed frames, generating non-saturated HDR radiance maps. Compared to existing methods on a novel dataset of real indoor scenes captured with our apparatus and including HDR ground truth lighting, PanDORA achieves superior visual fidelity and provides a scalable solution for capturing real environments in HDR.

Figures (10)

• Figure 1: Close-up of our proposed capture apparatus. Two Ricoh Theta Z1 panoramic cameras are rigidly attached to a portable monopod, allowing for easy manipulation. One camera captures a well-exposed video, while the other uses a much faster exposure to properly image the light sources.
• Figure 2: PanDORA NeRF architecture, where we split the training of both well- and fast-exposed images into their own, separate MLPs. The networks are trained in two stages: in the first stage, only the well-exposed and density MLP networks are trained. In the second stage, the fast-exposed MLP is trained while the well-exposed MLP is finetuned.
• Figure 3: HDR image generation by combining the well- and fast-exposed images from PanDORA. The predicted pixel values $\mathbf{z}_\mathrm{w}$ and $\mathbf{z}_\mathrm{f}$ are first linearized the inverse CRF $f^{-1}$, then combined by using weighting functions $w_\mathrm{w}$ and $w_\mathrm{f}$, resulting in an HDR image (right, under-exposed and tonemapped for visualization).
• Figure 4: Importance of the fine alignment. Relying solely on the camera pose estimation (see \ref{['sec:preprocessing']}) may cause slight misalignments between the well- and fast-exposed frames (third column), resulting in visual artifacts in the NeRF reconstruction. Our proposed fine alignment (see \ref{['sec:fine_alignment']}) corrects for this and reduces the visual artifacts.
• Figure 5: Representative images of the 14 scenes captured in our evaluation dataset. The numbers between parentheses indicate each scene's corresponding exposure factor $1/\Delta t_\mathrm{f}$, i.e., the relative factor between the chosen well and fast exposures.