SUCRe: Leveraging Scene Structure for Underwater Color Restoration

TL;DR

This paper proposes two novel methods that rely on different sets of inputs to recover the original colors of the scene as if the water had no effect on them, and introduces SUCRe, a new method that further exploits the scene's 3D Structure for Underwater Color Restoration.

Abstract

Underwater images are altered by the physical characteristics of the medium through which light rays pass before reaching the optical sensor. Scattering and wavelength-dependent absorption significantly modify the captured colors depending on the distance of observed elements to the image plane. In this paper, we aim to recover an image of the scene as if the water had no effect on light propagation. We introduce SUCRe, a novel method that exploits the scene's 3D structure for underwater color restoration. By following points in multiple images and tracking their intensities at different distances to the sensor, we constrain the optimization of the parameters in an underwater image formation model and retrieve unattenuated pixel intensities. We conduct extensive quantitative and qualitative analyses of our approach in a variety of scenarios ranging from natural light to deep-sea environments using three underwater datasets acquired from real-world scenarios and one synthetic dataset. We also compare the performance of the proposed approach with that of a wide range of existing state-of-the-art methods. The results demonstrate a consistent benefit of exploiting multiple views across a spectrum of objective metrics. Our code is publicly available at https://github.com/clementinboittiaux/sucre.

Figures (25)

• Figure 1: Multi-view tracking. We track pixels in multiple images to retrieve their intensities at different distances. We then estimate simultaneously their corrected color and the parameters of an underwater image formation model.
• Figure 2: SUCRe pipeline. We use camera poses, intrinsics and depth maps resulting from a SfM to pair geometrically pixels between different views. We project pixels from one view to another, enabling us to pair points in low contrast areas. We then simultaneously estimate an UIFM parameters along with the restored image. This figure illustrates our method on a real-world deep-sea dive at a submarine wreck.
• Figure 3: Applying SUCRe on a deep-sea image from the Eiffel Tower dataset matabos2015eiffelboittiaux2023eiffeltower captured by a ROV equipped with an artificial lighting system. The figure depicts the recovery of colors in low contrast areas (top left of the image). Pixels without depth information are left blank.
• Figure 4: Texturing the Eiffel Tower hydrothermal vent 3D model with images restored using our method results in a final model with improved visual quality, including finer details and more accurate colors compared to the original model.
• Figure 5: Chart $\pmb{\bar{\psi}}$ error vs. distance. Compared to other methods, SUCRe demonstrates lower and more consistent $\bar{\psi}$ errors independently of the distance of the color chart.