Phaseformer: Phase-based Attention Mechanism for Underwater Image Restoration and Beyond

TL;DR

Phaseformer addresses underwater image restoration by leveraging a phase-based transformer that emphasizes structure via phase information. It introduces a Phase-based Transformer Block with phase-derived queries/keys and an Optimized Phase Attention Block to propagate encoder features through phase-aware skip connections, coupled with learnable loss weights to balance multiple objectives. The approach achieves strong performance on synthetic and real underwater datasets while remaining lightweight (approximately $1.77\times 10^6$ parameters and $\sim13.0\times 10^9$ FLOPs), and also demonstrates applicability to low-light enhancement and downstream vision tasks. This combination of efficiency, phase-informed attention, and adaptive optimization yields practical benefits for autonomous underwater systems and related image enhancement domains.

Abstract

Quality degradation is observed in underwater images due to the effects of light refraction and absorption by water, leading to issues like color cast, haziness, and limited visibility. This degradation negatively affects the performance of autonomous underwater vehicles used in marine applications. To address these challenges, we propose a lightweight phase-based transformer network with 1.77M parameters for underwater image restoration (UIR). Our approach focuses on effectively extracting non-contaminated features using a phase-based self-attention mechanism. We also introduce an optimized phase attention block to restore structural information by propagating prominent attentive features from the input. We evaluate our method on both synthetic (UIEB, UFO-120) and real-world (UIEB, U45, UCCS, SQUID) underwater image datasets. Additionally, we demonstrate its effectiveness for low-light image enhancement using the LOL dataset. Through extensive ablation studies and comparative analysis, it is clear that the proposed approach outperforms existing state-of-the-art (SOTA) methods.

Figures (8)

• Figure 1: t-SNE visualization of the Amplitude and Phase of clean and degraded images. The separate clusters for clean and degraded amplitude show that there is more effect of degradation on amplitude content as compared to phase content which has overlapping clusters for clean and noisy images.
• Figure 2: Architectural schematic of the proposed underwater image enhancement network. The network comprises of Phase-based Transformer Block, Optimized Phase Attention Block, and dynamic weight assignment to the loss functions. The proposed phase-based transformer block captures local and non-local structural information using less attenuated phase-based queries and keys. Optimized Phase Attention block is proposed for propagating prominent attentive features from the encoder to the respective decoder. The dynamic weight ($\Omega$) assignment is proposed for efficient optimization of the network while training.
• Figure 3: Qualitative comparison of the proposed method (Ours) with existing SOTA methods for underwater image restoration on the synthetic datasets (upper part: UIEB dataset, lower part: UFO-120).
• Figure 4: Qualitative comparison of the proposed method (Ours) with existing SOTA methods for underwater image restoration on real-world UCCS, U45, and SQUID datasets.
• Figure 5: Qualitative comparison of results obtained with various loss settings.