Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption

TL;DR

Aleth-NeRF addresses the inability of standard NeRF to render accurately under extreme lighting by introducing Concealing Fields that modulate light transport between the scene and the observer. It jointly learns a volumetric representation and light-concealing factors (local $\Omega$ and global $\Theta_G$) with an unsupervised loss suite, enabling end-to-end restoration of normal-light views from low-light and over-exposed inputs. The approach is evaluated on the Challenging Illumination Multi-view (LOM) dataset, where Aleth-NeRF achieves state-of-the-art or competitive results in novel-view synthesis under adverse lighting, outperforming conventional NeRF with post-hoc enhancements and several HDR/RAW baselines. This method advances 3D scene understanding in non-ideal illumination and offers a practical, end-to-end framework for light-aware view synthesis, with potential extensions to broader lighting and nonuniform conditions.

Abstract

The standard Neural Radiance Fields (NeRF) paradigm employs a viewer-centered methodology, entangling the aspects of illumination and material reflectance into emission solely from 3D points. This simplified rendering approach presents challenges in accurately modeling images captured under adverse lighting conditions, such as low light or over-exposure. Motivated by the ancient Greek emission theory that posits visual perception as a result of rays emanating from the eyes, we slightly refine the conventional NeRF framework to train NeRF under challenging light conditions and generate normal-light condition novel views unsupervised. We introduce the concept of a "Concealing Field," which assigns transmittance values to the surrounding air to account for illumination effects. In dark scenarios, we assume that object emissions maintain a standard lighting level but are attenuated as they traverse the air during the rendering process. Concealing Field thus compel NeRF to learn reasonable density and colour estimations for objects even in dimly lit situations. Similarly, the Concealing Field can mitigate over-exposed emissions during the rendering stage. Furthermore, we present a comprehensive multi-view dataset captured under challenging illumination conditions for evaluation. Our code and dataset available at https://github.com/cuiziteng/Aleth-NeRF

Figures (10)

• Figure 1: Utilizing the Concealing Field assumption, Aleth-NeRF is capable of processing both low-light $\&$ over-expose multi-view images as inputs and generating novel views with natural illumination.
• Figure 2: Train on adverse lighting condition images $C^{adv}$, Aleth-NeRF performs unsupervised lightness correction by (a). remove concealing fields in low-light conditions and (b). add concealing fields in over-exposure conditions.
• Figure 3: Along camera ray r ($z$ axis), concealing fields and density $\sigma$ exhibit a negative correlation, $(x, y)$ denotes training images' width and height.
• Figure 4: A low-light "bike" scene for example, we show the ablation analyze of different loss functions' effectiveness.
• Figure 5: An comparison of enhancement results and model efficiency with RAW-NeRF raw_nerf, note that RAW-NeRF take 16-bits HDR image as inputs while NeRF $\&$ Aleth-NeRF take 8-bits LDR image as inputs.