Underwater Image Restoration Through a Prior Guided Hybrid Sense Approach and Extensive Benchmark Analysis

TL;DR

Underwater imaging suffers color casts and blur due to wavelength-dependent attenuation. The paper introduces GuidedHybSensUIR, a Color-Balance-Prior guided Hybrid Sense UIR framework that fuses a CNN-based Detail Restorer with a Transformer-based Feature Contextualizer and a Scale Harmonizer for robust multi-scale restoration, guided by a Gray World inspired color prior. The color prior is defined as $Prior_i(x,y) = \frac{R(x,y)+G(x,y)+B(x,y)}{3}$ and is integrated into cross-attention and final fusion, while the loss combines $L = w_1 L_f + w_2 L_s + w_3 L_p$ with $w_1=1$, $w_2=0.3$, $w_3=0.7$, to balance pixel fidelity, structural similarity, and perceptual quality. A large, standardized benchmark built from UIEB, EUVP, LSUI, and RUIE enables fair comparison against 37 baselines, with results showing state-of-the-art performance across paired and unpaired datasets. The work advances practical underwater image restoration by delivering superior color correction and detail preservation, along with publicly available code and data for reproducibility and benchmarks.

Abstract

Underwater imaging grapples with challenges from light-water interactions, leading to color distortions and reduced clarity. In response to these challenges, we propose a novel Color Balance Prior \textbf{Guided} \textbf{Hyb}rid \textbf{Sens}e \textbf{U}nderwater \textbf{I}mage \textbf{R}estoration framework (\textbf{GuidedHybSensUIR}). This framework operates on multiple scales, employing the proposed \textbf{Detail Restorer} module to restore low-level detailed features at finer scales and utilizing the proposed \textbf{Feature Contextualizer} module to capture long-range contextual relations of high-level general features at a broader scale. The hybridization of these different scales of sensing results effectively addresses color casts and restores blurry details. In order to effectively point out the evolutionary direction for the model, we propose a novel \textbf{Color Balance Prior} as a strong guide in the feature contextualization step and as a weak guide in the final decoding phase. We construct a comprehensive benchmark using paired training data from three real-world underwater datasets and evaluate on six test sets, including three paired and three unpaired, sourced from four real-world underwater datasets. Subsequently, we tested 14 traditional and retrained 23 deep learning existing underwater image restoration methods on this benchmark, obtaining metric results for each approach. This effort aims to furnish a valuable benchmarking dataset for standard basis for comparison. The extensive experiment results demonstrate that our method outperforms 37 other state-of-the-art methods overall on various benchmark datasets and metrics, despite not achieving the best results in certain individual cases. The code and dataset are available at \href{https://github.com/CXH-Research/GuidedHybSensUIR}{https://github.com/CXH-Research/GuidedHybSensUIR}.

Figures (11)

• Figure 1: Color distribution visualized by 3D scatter plot. (a), (b), and (c) represent the individual color scatter plots of the input underwater image, color balance prior, and the output restored image of our model, respectively. To enhance comparison, these scatter plots are amalgamated in (e). The corresponding input and output images are presented in (d) and (f) for visual reference.
• Figure 2: Overview of the GuidedHybSensUIR architecture—a U-shaped model with encoder, bottleneck, and decoder stages enabling hybrid sense-level feature processing. Detail Restorer modules in the decoder enhance local details at three scales. At the bottleneck, the Feature Contextualizer—guided by the color balance prior—captures long-range relationships and global color dependencies for effective color correction and enhancement. Scale Harmonizer modules in the decoder refine fused information after each fusion step, ensuring seamless multi-scale integration for high-quality image reconstruction.
• Figure 3: Architecture of the Nonlinear Activation-Free Block (NAFB).
• Figure 4: Illustration of the ContextBlock within the Residual Context Block (RCB).
• Figure 5: Architecture of the Feature Contextualizer module. The $3\times3$ convolutional "embed" block projects the $12$ input channels to $48$ embedding channels. The module contains four Multi-Attention Quaternion (MAQ) blocks, each composed of three types of inter-channel attention transformers: the Adjust Color Transformer (ACT), the Keep Feature Transformer (KFT), and the Self-Attention Transformer (SAT). The parallel outputs of these transformers are fused using quaternion convolution.