Removing Adverse Volumetric Effects From Trained Neural Radiance Fields

TL;DR

It is argued that the traditional NeRF models are able to replicate scenes filled with fog and a method to remove the fog when synthesizing novel views is proposed, by calculating the global contrast of a scene to estimate a density threshold that, when applied, removes all visible fog.

Abstract

While the use of neural radiance fields (NeRFs) in different challenging settings has been explored, only very recently have there been any contributions that focus on the use of NeRF in foggy environments. We argue that the traditional NeRF models are able to replicate scenes filled with fog and propose a method to remove the fog when synthesizing novel views. By calculating the global contrast of a scene, we can estimate a density threshold that, when applied, removes all visible fog. This makes it possible to use NeRF as a way of rendering clear views of objects of interest located in fog-filled environments. Additionally, to benchmark performance on such scenes, we introduce a new dataset that expands some of the original synthetic NeRF scenes through the addition of fog and natural environments. The code, dataset, and video results can be found on our project page: https://vegardskui.com/fognerf/

Figures (6)

• Figure 1: Generic NeRF model trained on hazy data, before (top left) and after (bottom right) applying our algorithm.
• Figure 2: The ground truth images compared with the NeRF synthesized view of the same images from the test set. Note that the dark images on the far right are not an error but rather the effect of heavy fog in these lighting conditions.
• Figure 3: The overview of our model, where a set of pixel rays $\mathcal{R}$ are sampled from a converged NeRF model that has been trained on foggy images. Multiple sets of colors are derived from $\mathcal{R}$ as a function of different density threshold values $\sigma_\text{thre}$ where the colors are then converted to luminance in order to estimate a global contrast of the radiance field. The global contrast as a function of density threshold values $\sigma_\text{thre}$ is used to find the converging point, where this point will be used as a density threshold when synthesizing new views.
• Figure 4: The densities $\sigma$ along a pixel ray $\vec{r}(t)$ in a theoretical scenario where there is a constant density $\sigma_\text{fog}$ for fog. Due to fog being transparent, its density will be a magnitude lower than the densities associated with solid / non-transparent volume.
• Figure 5: Sample densities along a single pixel ray, corresponding to the central ray of the middle image in fig. \ref{['subfig:thresholding_results_desert']}.