WaterHE-NeRF: Water-ray Tracing Neural Radiance Fields for Underwater Scene Reconstruction

TL;DR

WaterHE-NeRF addresses underwater NeRF limitations caused by light attenuation by introducing a water-ray tracing field based on Retinex theory to model illuminance attenuation alongside color and density. It synthesizes both degraded and restored underwater views and optimizes restoration via a Wasserstein distance to histogram-equalized pseudo-GT, with batch-averaging to preserve color distributions. Trained on real UWBundle and synthetic LLFF-Water data, the approach demonstrates improved multi-view restoration quality and consistency over state-of-the-art methods. The framework advances 3D-consistent underwater image reconstruction, offering practical gains for underwater robotics and immersive viewing, with code promised publicly.

Abstract

Neural Radiance Field (NeRF) technology demonstrates immense potential in novel viewpoint synthesis tasks, due to its physics-based volumetric rendering process, which is particularly promising in underwater scenes. Addressing the limitations of existing underwater NeRF methods in handling light attenuation caused by the water medium and the lack of real Ground Truth (GT) supervision, this study proposes WaterHE-NeRF. We develop a new water-ray tracing field by Retinex theory that precisely encodes color, density, and illuminance attenuation in three-dimensional space. WaterHE-NeRF, through its illuminance attenuation mechanism, generates both degraded and clear multi-view images and optimizes image restoration by combining reconstruction loss with Wasserstein distance. Additionally, the use of histogram equalization (HE) as pseudo-GT enhances the network's accuracy in preserving original details and color distribution. Extensive experiments on real underwater datasets and synthetic datasets validate the effectiveness of WaterHE-NeRF. Our code will be made publicly available.

Figures (7)

• Figure 1: WaterHE-NeRF is formulated to build a radiance field with illuminance attenuation by taking multi-view underwater images as input. After training is completed, WaterHE-NeRF can synthesize novel underwater views, and synthesize their corresponding restored views by removing illuminance attenuation.
• Figure 2: The overview of WaterHE-NeRF. Specifically, the model takes multi-view degraded underwater images as input, synthesizing underwater novel views and restored novel views. Then the reconstruction loss is computed between underwater views and raw input, while the color distribution loss is computed between restored views and histogram-equalized images.
• Figure 3: The pipeline of WaterHE-NeRF. The sample points on the input ray go through the MLP network to predict the pixel value, density, and illuminance attenuation of each point. In the testing stage, the illuminance attenuation is removed to rerender restored images. Our method takes the image after HE as a pseudo GT value and learns the color distribution from it to guide the restored underwater image to have the correct color distribution.
• Figure 4: Scene rendering and enhanced images on the UWbundle dataset. Left to right: Raw input, Histogram Equalization pizer1987adaptive, FA+ jiang2023five, CIFM+ zhou2023underwater, FUNIE-GAN islam2020fast, Seathru-NeRF levy2023seathru, Neural-Sea zhang2023beyond, our method without batch-averaging, our WaterHE-NeRF result. Top to bottom: novel views and corresponding enhanced images.
• Figure 5: Experiment on synthetic Horns scene of the LLFF-water dataset. From top to bottom, we present images from different perspectives in both the training stage and the testing stage using unsupervised methods, including HE pizer1987adaptive, CIFM+ zhou2023underwater, FUNIE-GAN islam2020fast, Seathru-NeRF levy2023seathru, Neural-Sea zhang2023beyond, our methods without batch-averaging, and our completed method.