Toward Sufficient Spatial-Frequency Interaction for Gradient-aware Underwater Image Enhancement

TL;DR

Underwater images suffer from color distortion and scattering; the paper addresses this by exploiting Fourier-domain information, arguing that degradation primarily lies in the amplitude component. It introduces SFGNet, a two-stage framework consisting of DSFFNet for dense spatial-frequency fusion and a gradient-aware corrector (GAC) that uses a gradient map to refine edges and textures. The DSFFNet employs dense Fourier fusion blocks (DFF) and dense spatial fusion blocks (DSF) with cross connections, and constrains the amplitude in Fourier space via $\mathcal{L}_{s1}$; the GAC uses a curriculum strategy to adaptively fuse gradient-based guidance through $\lambda$ and optimizes with $\mathcal{L}_g$ and a perceptual term via a pre-trained network $\phi$. The approach achieves competitive state-of-the-art results on UIEBD and LSUI datasets, indicating that combining spatial-frequency interaction and gradient guidance improves both objective metrics and visual quality; code is available at GitHub.

Abstract

Underwater images suffer from complex and diverse degradation, which inevitably affects the performance of underwater visual tasks. However, most existing learning-based Underwater image enhancement (UIE) methods mainly restore such degradations in the spatial domain, and rarely pay attention to the fourier frequency information. In this paper, we develop a novel UIE framework based on spatial-frequency interaction and gradient maps, namely SFGNet, which consists of two stages. Specifically, in the first stage, we propose a dense spatial-frequency fusion network (DSFFNet), mainly including our designed dense fourier fusion block and dense spatial fusion block, achieving sufficient spatial-frequency interaction by cross connections between these two blocks. In the second stage, we propose a gradient-aware corrector (GAC) to further enhance perceptual details and geometric structures of images by gradient map. Experimental results on two real-world underwater image datasets show that our approach can successfully enhance underwater images, and achieves competitive performance in visual quality improvement. The code is available at https://github.com/zhihefang/SFGNet.

Figures (6)

• Figure 1: Our motivations. The amplitude and phase are produced by Fast Fourier Transform (FFT) and the compositional images are obtained by Inverse FFT (IFFT). We swap the amplitude components of the underwater image and its corresponding label image. The recombined result of the amplitude of label image and the phase of underwater image has similar visual appearance with label image.
• Figure 2: Overall framework of our proposed SFGNet. SFGNet mainly consists of a dense spatial-frequency fusion network (DSFFNet) and gradient-aware corrector (GAC). In GAC, CL refers to the curricular learning strategy. DSFFNet aims to achieve sufficient spatial-frequency interaction by cross connections between dense fourier fusion (DFF) block and dense spatial fusion (DSF) block. To further refine the details, we propose a novel gradient-aware corrector (GAC) to further enhance perceptual details and geometric structures of images by gradient map.
• Figure 3: The detailed structure of the designed DFF block.
• Figure 4: The detailed structure of the designed DSF block.
• Figure 5: Comparisons of visual results with all baselines on the UIEBD dataset. Red box marked image is a zoomed image.