Measuring and Controlling the Spectral Bias for Self-Supervised Image Denoising

TL;DR

The paper addresses spectral bias in self-supervised denoising for paired noisy images by proposing SCNet, a framework that decouples high-frequency structure from noise in the frequency domain. It introduces Image Pair Frequency-Band Similarity (IPFS) to quantify spectral alignment between outputs, mapped noisy inputs, and ground truth, and couples this with Frequency Selection Decision (FSD), Lipschitz-constrained convolution, and a Spectral Separation and low-rank Reconstruction (SSR) module. The key contributions are (i) a frequency-based selection strategy to accelerate convergence, (ii) a Lipschitz-based constraint to limit high-frequency learning, and (iii) a differentiable SSR that preserves high-frequency textures via a low-rank reconstruction, validated on synthetic and real datasets. The results show SCNet achieves competitive performance with state-of-the-art unsupervised methods and even surpasses some supervised baselines in certain settings, offering practical gains for denoising in real imaging pipelines.

Abstract

Current self-supervised denoising methods for paired noisy images typically involve mapping one noisy image through the network to the other noisy image. However, after measuring the spectral bias of such methods using our proposed Image Pair Frequency-Band Similarity, it suffers from two practical limitations. Firstly, the high-frequency structural details in images are not preserved well enough. Secondly, during the process of fitting high frequencies, the network learns high-frequency noise from the mapped noisy images. To address these challenges, we introduce a Spectral Controlling network (SCNet) to optimize self-supervised denoising of paired noisy images. First, we propose a selection strategy to choose frequency band components for noisy images, to accelerate the convergence speed of training. Next, we present a parameter optimization method that restricts the learning ability of convolutional kernels to high-frequency noise using the Lipschitz constant, without changing the network structure. Finally, we introduce the Spectral Separation and low-rank Reconstruction module (SSR module), which separates noise and high-frequency details through frequency domain separation and low-rank space reconstruction, to retain the high-frequency structural details of images. Experiments performed on synthetic and real-world datasets verify the effectiveness of SCNet.

Figures (9)

• Figure 2: Spectrum measurement of paired noisy images during network iteration in the existing method. (a) represents the frequency band similarity between the output and the mapped noisy image; (b) represents the frequency band similarity between the output and the ground truth (GT). The peak value of PSNR is indicated by the dashed line.
• Figure 3: (a) Overall architecture of the SCNet. (b) Perform frequency domain selection on noisy image pairs using FSD. (c) Control convolutional layers to constrain high-frequency noise by using the Lipschitz constant. (d) Separate and reconstruct high-frequency details by using the SSR module.
• Figure 4: Visual comparison of denoising sRGB images in the setting of $\sigma=25$.
• Figure 5: Visual comparison of denoising FM images.
• Figure 6: The denoising performance of the Lipschitz method on top 3 frequency bands ($\bar{S}_{W^{(t)} \rightarrow y_{nos}}$, similarity between the output and the noisy image).