Wavelet-based Mamba with Fourier Adjustment for Low-light Image Enhancement

TL;DR

This work proposes a novel Wavelet-based Mamba with Fourier Adjustment model called WalMaFa, consisting of a Wavelet-based Mamba Block (WMB) and a Fast Fourier Adjustment Block (FFAB) and employs an Encoder-Latent-Decoder structure to accomplish the end-to-end transformation.

Abstract

Frequency information (e.g., Discrete Wavelet Transform and Fast Fourier Transform) has been widely applied to solve the issue of Low-Light Image Enhancement (LLIE). However, existing frequency-based models primarily operate in the simple wavelet or Fourier space of images, which lacks utilization of valid global and local information in each space. We found that wavelet frequency information is more sensitive to global brightness due to its low-frequency component while Fourier frequency information is more sensitive to local details due to its phase component. In order to achieve superior preliminary brightness enhancement by optimally integrating spatial channel information with low-frequency components in the wavelet transform, we introduce channel-wise Mamba, which compensates for the long-range dependencies of CNNs and has lower complexity compared to Diffusion and Transformer models. So in this work, we propose a novel Wavelet-based Mamba with Fourier Adjustment model called WalMaFa, consisting of a Wavelet-based Mamba Block (WMB) and a Fast Fourier Adjustment Block (FFAB). We employ an Encoder-Latent-Decoder structure to accomplish the end-to-end transformation. Specifically, WMB is adopted in the Encoder and Decoder to enhance global brightness while FFAB is adopted in the Latent to fine-tune local texture details and alleviate ambiguity. Extensive experiments demonstrate that our proposed WalMaFa achieves state-of-the-art performance with fewer computational resources and faster speed. Code is now available at: https://github.com/mcpaulgeorge/WalMaFa.

Figures (8)

• Figure 1: WalMaFa consistently achieves relatively better performance and less computing complexity on LOL-v2-syn dataset. WalMaFa also stands out in human-perceived user ratings with 15 participants.
• Figure 1: The visual comparisons with coarse-to-fine pipeline.
• Figure 2: The motivation of WalMaFa. The SSIM error map between the output after switching different components and the low/high image shows that the low-frequency LL component of 2D Discrete Wavelet Transform is more sensitive to global color brightness, while the phase component of Fast Fourier Transform is more sensitive to local texture detail information.
• Figure 3: The overview of WalMaFa architecture. Our model consists of an Encoder-Latent-Decoder structure that uses wavelet-based WMB to adjust global brightness during the Encoder and Decoder, and Fourier-based FFAB to adjust local details during the Latent.
• Figure 4: The illustration of Wavelet-based Mamba Block (WMB) and Fast Fourier Adjustment Block (FFAB).