Advanced Underwater Image Restoration in Complex Illumination Conditions

TL;DR

This work presents a general underwater image formation framework and a 3D frustum-based lookup table that stores per-voxel $\alpha$ and $\beta$ for each color channel to model integrated lighting and backscatter in scenes illuminated by co-moving artificial sources. By leveraging Lambertian shading, Known Color, Correspondence, Smooth, and Pure Water constraints, it enables robust, fast restoration of true object albedo across complex illumination, with a hierarchical estimation strategy and carefully designed weights. Extensive simulations and real-world experiments demonstrate effective suppression of lighting patterns and water effects, improving texture fidelity for large-scale 3D mapping and photogrammetry, and the method adapts to in-air scenarios with artificial illumination. The approach offers a practical, calibration-light pathway to high-fidelity underwater texture reconstruction in challenging environments.

Abstract

Underwater image restoration has been a challenging problem for decades since the advent of underwater photography. Most solutions focus on shallow water scenarios, where the scene is uniformly illuminated by the sunlight. However, the vast majority of uncharted underwater terrain is located beyond 200 meters depth where natural light is scarce and artificial illumination is needed. In such cases, light sources co-moving with the camera, dynamically change the scene appearance, which make shallow water restoration methods inadequate. In particular for multi-light source systems (composed of dozens of LEDs nowadays), calibrating each light is time-consuming, error-prone and tedious, and we observe that only the integrated illumination within the viewing volume of the camera is critical, rather than the individual light sources. The key idea of this paper is therefore to exploit the appearance changes of objects or the seafloor, when traversing the viewing frustum of the camera. Through new constraints assuming Lambertian surfaces, corresponding image pixels constrain the light field in front of the camera, and for each voxel a signal factor and a backscatter value are stored in a volumetric grid that can be used for very efficient image restoration of camera-light platforms, which facilitates consistently texturing large 3D models and maps that would otherwise be dominated by lighting and medium artifacts. To validate the effectiveness of our approach, we conducted extensive experiments on simulated and real-world datasets. The results of these experiments demonstrate the robustness of our approach in restoring the true albedo of objects, while mitigating the influence of lighting and medium effects. Furthermore, we demonstrate our approach can be readily extended to other scenarios, including in-air imaging with artificial illumination or other similar cases.

Figures (19)

• Figure 1: Different image formation models under different illumination conditions.
• Figure 2: Examples of underwater seafloor images captured under different illumination conditions, each corresponding to a different image formation model. I: In surface water where the strong sunlight creates a dynamic caustic pattern. II: In shallow water where the illumination is relatively homogeneous due to the abundant sunlight. III: in the twilight zone where the sunlight is severely attenuated and additional artificial light is used to illuminate the scene. IV: In complete darkness in the depth ocean and is illuminated solely by artificial light sources.
• Figure 3: Two popular underwater image formation models used in underwater image restoration. Top: Shallow water image formation with homogenous illumination from the sunlight. Bottom: Deep water image formation under artificial illumination.
• Figure 4: Proposed 3D lookup table structure. The camera viewing frustum is sliced into several slabs and each slab is constructed by a plane of voxels. Each voxel with in a slab stores two parameters: a multiplicative factor $\alpha$ and a additive factor $\beta$, for each color channel. These parameters represent the combined effect of lighting and water at that particular 3D position. Giving the stable lighting and water conditions during a single mission, either under homogeneous illumination in shallow water or co-moving artificial light source in deep water, the parameters in the lookup table are relatively fixed, enabling rapid batch restoration of entire image sequences.
• Figure 5: One observed color ($I$) with a known color ($I_0$) can only provide a constraint on $\alpha$ and $\beta$ along a line in the $\alpha$-$\beta$ plane. To obtain a unique solution for each voxel, at least two observations with different known colors are required. As shown in the figure, the blue line is the constraint from one observed underwater color $I_1$ at voxel $V_i$ with known color $I_0$, while the red line refers to the constraint from another underwater color observation $I_2$ at the same voxel with second known color $I_0'$. The intersection point of the two lines (in green) provides the unique solution $(\alpha_i,\beta_i)$ for voxel $V_i$. Due to the uncertainty $\sigma$ in the observations, each line is only constrained in the green interval and the ambiguity of the solution is defined by the intersection of the two constraint regions (in yellow). To minimize this ambiguity and reduce the uncertainty of the solution, slopes of two lines ($-I_0$ and $-I_0'$, respectively) should be perpendicular to each other in order to achieve minimum intersection of intervals. Therefore, two known colors with widely disparate values should be used for the observations.