Aleth-NeRF: Low-light Condition View Synthesis with Concealing Fields

TL;DR

The proposed method, Aleth-NeRF, directly learns from the dark image to understand volumetric object representation and concealing field under priors and can render a single or multi-view well-lit image(s) and gain superior performance over other 2D low-light enhancement methods.

Abstract

Common capture low-light scenes are challenging for most computer vision techniques, including Neural Radiance Fields (NeRF). Vanilla NeRF is viewer-centred simplifies the rendering process only as light emission from 3D locations in the viewing direction, thus failing to model the low-illumination induced darkness. Inspired by the emission theory of ancient Greeks that visual perception is accomplished by rays casting from eyes, we make slight modifications on vanilla NeRF to train on multiple views of low-light scenes, we can thus render out the well-lit scene in an unsupervised manner. We introduce a surrogate concept, Concealing Fields, that reduces the transport of light during the volume rendering stage. Specifically, our proposed method, Aleth-NeRF, directly learns from the dark image to understand volumetric object representation and concealing field under priors. By simply eliminating Concealing Fields, we can render a single or multi-view well-lit image(s) and gain superior performance over other 2D low-light enhancement methods. Additionally, we collect the first paired LOw-light and normal-light Multi-view (LOM) datasets for future research. This version is invalid, please refer to our new AAAI version: arXiv:2312.09093

Figures (8)

• Figure 1: We assume objects are naturally visible. However, the Concealing Field attenuates the light in the viewing direction, making the left user see a low-light scene. Aleth-NeRF takes a low-light image as input and unsupervisly learns the distribution of the Concealing Field. Then, we unconceal (alethia) the Concealing field to render the enhanced image. This scene is taken from LOM dataset.
• Figure 2: (a). NeRF rendering results in normal-light scene and low-light scene. (b). NeRF rendering on enhanced scene by 2D image enhancement methods LIME LIME and IAT BMVC22_IAT. (c) Aleth-NeRF rendering results in low-light scene.
• Figure 3: Overview of the Aleth-NeRF architecture. Local $\Omega$ and Global $\Theta_G$ Concealing Fields are additionally learned and integrated into the NeRF framework. We use a modified volume rendering function to render low-light scene taking the Concealing Fields into account.
• Figure 4: Ablation analyze on the LOL dataset RetiNexNet, larger convolution size to generate local concealing field $\Omega$ would further improve enhanced scene $\hat{\textbf{C}}^{nor}$'s smoothness.
• Figure 5: Aleth-NeRF uses reconstruction loss ($\mathcal{L}_{nerf}$) on the low-light scene $\textbf{C}^{low}$, meanwhile additional constraints (control loss $\mathcal{L}_{con}$, structure loss $\mathcal{L}_{st}$ and color constancy loss $\mathcal{L}_{cc}$) are added to regularize the predicted normal-light scene $\hat{\textbf{C}}^{nor}$.