A Semi-supervised Physics-Aware Triple-Stream Underwater Image Enhancement Network

TL;DR

This work addresses underwater image degradation by integrating a revised Image Formation Model (IFM) with a physics-aware deep network. It proposes PATS-UIENet, a triple-stream architecture (D-Stream, B-Stream, A-Stream) that explicitly estimates $t_D^c(x,oldsymbol{D})$, $t_B^c(x,oldsymbol{B})$, and $A^c$, enabling physically grounded enhancement. An IFM-inspired semi-supervised framework leverages both labeled and unlabeled data through bi-directional supervision and unsupervised degradation to improve generalization. Experiments on five datasets show state-of-the-art or competitive performance across full-reference and non-reference metrics, with ablations confirming the benefits of the semi-supervised framework and the RCT/RCM modules for robust color restoration and transmission estimation.

Abstract

Underwater images normally suffer from degradation due to the transmission medium of water bodies. Both traditional prior-based approaches and deep learning-based methods have been used to address this problem. However, the inflexible assumption of the former often impairs their effectiveness in handling diverse underwater scenes, while the generalization of the latter to unseen images is usually weakened by insufficient data. In this study, we leverage both the physics-based Image Formation Model (IFM) and deep learning techniques for Underwater Image Enhancement (UIE). To this end, we propose a novel Physics-Aware Triple-Stream Underwater Image Enhancement Network, i.e., PATS-UIENet, which comprises a Direct Signal Transmission Estimation Stream (D-Stream), a Backscatter Signal Transmission Estimation Stream (B-Stream) and an Ambient Light Estimation Stream (A-Stream). This network fulfills the UIE task by explicitly estimating the degradation parameters of a revised IFM. We also adopt an IFM-inspired semi-supervised learning framework, which exploits both the labeled and unlabeled images, to address the issue of insufficient data. To our knowledge, such a physics-aware deep network and the IFM-inspired semi-supervised learning framework have not been used for the UIE task before. Our method performs better than, or at least comparably to, sixteen baselines across four testing sets in the degradation estimation and UIE tasks. These promising results should be due to the fact that the proposed method can not only model the degradation but also learn the characteristics of diverse underwater scenes.

Figures (6)

• Figure 1: Examples of a degraded underwater image, the corresponding ambient light image $A^c$ and three transmission maps $t^c(x)$ ($c \in \{R,G,B\}$) estimated, and the enhanced image produced using the estimated parameters of the IFM IFM.
• Figure 2: The architecture of the proposed PATS-UIENet, which comprises three individual streams, namely, D-Stream, B-Stream and A-Stream, to estimate the degradation parameters of the revised IFM revised_IFM.
• Figure 3: The proposed semi-supervised learning framework, which comprises a bi-directional supervised learning scheme and an unsupervised learning scheme, used for training our PATS-UIENet.
• Figure 4: The results produced by 16 baselines and our method in terms of a degraded image in the Test-U80 testing set.
• Figure 5: The results produced by 16 baselines and our method in terms of a degraded image in the Test-S110 testing set.