Low-Trace Adaptation of Zero-shot Self-supervised Blind Image Denoising

TL;DR

Inspired by the properties of the Frobenius norm expansion, it is discovered that incorporating a trace term reduces the optimization goal disparity between self-supervised and supervised methods, thereby enhancing the performance of self-supervised learning.

Abstract

Deep learning-based denoiser has been the focus of recent development on image denoising. In the past few years, there has been increasing interest in developing self-supervised denoising networks that only require noisy images, without the need for clean ground truth for training. However, a performance gap remains between current self-supervised methods and their supervised counterparts. Additionally, these methods commonly depend on assumptions about noise characteristics, thereby constraining their applicability in real-world scenarios. Inspired by the properties of the Frobenius norm expansion, we discover that incorporating a trace term reduces the optimization goal disparity between self-supervised and supervised methods, thereby enhancing the performance of self-supervised learning. To exploit this insight, we propose a trace-constraint loss function and design the low-trace adaptation Noise2Noise (LoTA-N2N) model that bridges the gap between self-supervised and supervised learning. Furthermore, we have discovered that several existing self-supervised denoising frameworks naturally fall within the proposed trace-constraint loss as subcases. Extensive experiments conducted on natural and confocal image datasets indicate that our method achieves state-of-the-art performance within the realm of zero-shot self-supervised image denoising approaches, without relying on any assumptions regarding the noise.

Figures (6)

• Figure 1: Performance vs. training time on an RTX2080ti GPU. The results are evaluated on the McMaster18 dataset with gaussian noise $\sigma=10$. The red point represents our proposed network.
• Figure 2: Comparison of different denoising methods. Supervised denoising is trained using pairs of clean/noisy images. The Noise2Noise approach circumvents the need for clean samples by employing noisy-noisy image pairs. The Neighbor2Neighbor method further refines this by generating noisy-noisy pairs through the downsampling of a single noisy image. Our method takes a further step in the loss function by constraining the trace term. It guides the self-supervised model closer to the direction of supervised learning and yields superior performance without any prior assumptions about the noise model.
• Figure 3: The main pipeline of our proposed method. The two-stage model begins with a pretraining phase where the network is initially trained using an MSE loss, leading to a biased denoiser. To improve performance, the subsequent fine-tuning stage employs the trace-constrained loss that supplements the model's training beyond the MSE baseline. This two-step training process aims to narrow the gap between self-supervised and supervised learning techniques, thus enhancing the overall effectiveness of the model.
• Figure 4: Visual comparison between methods. Our proposed denoising approach demonstrates superior performance in preserving the fidelity of textural details, particularly in texture-rich regions, achieving the best denoising results compared to other methods.
• Figure 5: Visual comparison on confocal and medical datasets. Our approach maintains a greater level of detail within regions abundant in texture.