Zero-Shot CFC: Fast Real-World Image Denoising based on Cross-Frequency Consistency

TL;DR

The paper tackles real-world image denoising without training data or explicit noise models by introducing ZSCFC, which leverages cross-frequency texture consistency. It decomposes a noisy image into low- and high-frequency bands with an Image Multi-Frequency Decomposer (IMFD) and uses an ultralight texture extractor g to recover textures from the high-frequency bands, guided by the cross-frequency consistency losses $L_{cons1}$ and $L_{cons2}$ and a TV regularizer, combined as $L_{total} = \omega_1 L_{cons1} + \omega_2 L_{cons2} + \omega_3 L_{reg}$. The denoised image is reconstructed as $img_{denoised} = LFS_1 + g(HFS_1) + g(HFS_2) + g(HFS_3)$. Empirical results on six real-world datasets and synthetic pink-noise tests show ZSCFC achieves superior PSNR/SSIM while requiring only about 1.5k parameters and around 3 seconds per image, enabling efficient edge deployment and robust performance across diverse noise conditions.

Abstract

Zero-shot denoisers address the dataset dependency of deep-learning-based denoisers, enabling the denoising of unseen single images. Nonetheless, existing zero-shot methods suffer from long training times and rely on the assumption of noise independence and a zero-mean property, limiting their effectiveness in real-world denoising scenarios where noise characteristics are more complicated. This paper proposes an efficient and effective method for real-world denoising, the Zero-Shot denoiser based on Cross-Frequency Consistency (ZSCFC), which enables training and denoising with a single noisy image and does not rely on assumptions about noise distribution. Specifically, image textures exhibit position similarity and content consistency across different frequency bands, while noise does not. Based on this property, we developed cross-frequency consistency loss and an ultralight network to realize image denoising. Experiments on various real-world image datasets demonstrate that our ZSCFC outperforms other state-of-the-art zero-shot methods in terms of computational efficiency and denoising performance.

Figures (4)

• Figure 1: The overall architecture of ZSCFC.
• Figure 2: (left) Average LFS residual noise $\mathrm{std}$ at different $\mathrm{f_c}$. (right) The frequency bands of HFSs and LFSs.
• Figure 3: Above: An illustration of the output generated by the IMFD with $\mathrm{f_{c1}}$, $\mathrm{f_{c2}}$, and $\mathrm{f_{c3}}$. The $\mathrm{LFS_1}$, $\mathrm{HFS_1}$, $\mathrm{HFS_2}$, and $\mathrm{HFS_3}$ can be fused into the original noisy image. Below: Overview of our proposed IMFD framework.
• Figure 4: Visual quality comparison on SIDD Medium and Validation datasets.