CPDM: Content-Preserving Diffusion Model for Underwater Image Enhancement

TL;DR

CPDM first leverages a diffusion model as its fundamental model for stable training and then designs a content-preserving framework to deal with changes in imaging conditions, and outperforms state-of-the-art methods in both subjective and objective metrics.

Abstract

Underwater image enhancement (UIE) is challenging since image degradation in aquatic environments is complicated and changing over time. Existing mainstream methods rely on either physical-model or data-driven, suffering from performance bottlenecks due to changes in imaging conditions or training instability. In this article, we make the first attempt to adapt the diffusion model to the UIE task and propose a Content-Preserving Diffusion Model (CPDM) to address the above challenges. CPDM first leverages a diffusion model as its fundamental model for stable training and then designs a content-preserving framework to deal with changes in imaging conditions. Specifically, we construct a conditional input module by adopting both the raw image and the difference between the raw and noisy images as the input, which can enhance the model's adaptability by considering the changes involving the raw images in underwater environments. To preserve the essential content of the raw images, we construct a content compensation module for content-aware training by extracting low-level features from the raw images. Extensive experimental results validate the effectiveness of our CPDM, surpassing the state-of-the-art methods in terms of both subjective and objective metrics.

Figures (6)

• Figure 1: A visual illustration of the proposed CPDM method. (a) Original images to be enhanced. (b) Images enhanced by our CPDM. (c) Reference images as ground truth.
• Figure 2: Illustration of our conditional input module. The forward diffusion process denotes $q$ (from right to left), and the backward inference process denotes $p_{\theta_{cpdm}}$ (from left to right). $x_0$ and $y_0$ denote the clear and paired underwater images, respectively.
• Figure 3: Illustration of the proposed CPDM at time step $t$. Here, $y_0$ represents the to-be-enhanced underwater image, and $x_t$ denotes the noisy image of the current time step.
• Figure 4: Visual comparison of enhanced underwater images by various methods on the Test_L400 (LSUI) dataset. From left to right: original underwater image, results from WaterNet li2019underwater, FUnIE 9001231funie, Ucolor 9426457ucolor, U-shape Transformer peng2023u, our CPDM method, and the reference image.
• Figure 5: Visual comparison of enhanced underwater images by various methods on the Test_U90 (UIEB) dataset. From left to right: original underwater image, results from WaterNet li2019underwater, FUnIE 9001231funie, Ucolor 9426457ucolor, U-shape Transformer peng2023u, our CPDM method, and the reference image.