Learning to Translate Noise for Robust Image Denoising

TL;DR

This work tackles the generalization gap in image denoising for real-world noise by introducing a Noise Translation Network that maps arbitrary real noise to Gaussian noise, enabling use of Gaussian-denoising nets. The NTN is trained with implicit supervision (reconstructing the ground-truth image) and explicit supervision (matching translated noise to Gaussian noise in spatial and frequency domains via Wasserstein distances) and is built on Gaussian Injection Blocks to impose a Gaussian prior. Across multiple pretrained denoisers and real-noise benchmarks, the framework achieves consistent out-of-distribution gains, often substantially surpassing state-of-the-art methods, while maintaining test-time efficiency. The approach is model-agnostic, does not require test-time training, and provides detailed ablations and reproducibility resources, suggesting broad practical impact for robust real-world denoising.

Abstract

Deep learning-based image denoising techniques often struggle with poor generalization performance to out-of-distribution real-world noise. To tackle this challenge, we propose a novel noise translation framework that performs denoising on an image with translated noise rather than directly denoising an original noisy image. Specifically, our approach translates complex, unknown real-world noise into Gaussian noise, which is spatially uncorrelated and independent of image content, through a noise translation network. The translated noisy images are then processed by an image denoising network pretrained to effectively remove Gaussian noise, enabling robust and consistent denoising performance. We also design well-motivated loss functions and architectures for the noise translation network by leveraging the mathematical properties of Gaussian noise. Experimental results demonstrate that the proposed method substantially improves robustness and generalizability, outperforming state-of-the-art methods across diverse benchmarks. Visualized denoising results and the source code are available on our project page.

Figures (11)

• Figure 1: Denoising results of Gaussian-trained Restormer zamir2022restormer, applied to (a) original real noisy image, (b) noisy image with added Gaussian noise. We can observe that simply adding Gaussian noise to a real noisy image improves the denoising performance. (c) showcases our framework which translates real noise into Gaussian noise, enabling the denoising network to achieve even higher quality results. Zoom in for better details.
• Figure 2: Illustration of our training framework for the noise translation network.
• Figure 3: Comparison between the qualitative results of various denoising networks including ours (noise translation network with pretrained NAFNet), on the out-of-distribution (OOD) datasets. Our result displays cleaner outputs compared to other state-of-the-art networks trained on a real-noise dataset with self-supervised and supervised methods.
• Figure 4: Visual results of noise translation. The noisy image in the top row is from the Poly xu2018real dataset, while the one in the bottom row is from the CC nam2016holistic dataset. For enhanced visualization, noise is displayed as the absolute value, scaled by a factor of 10.
• Figure 5: Histogram of noise distribution before (Real) and after noise translation network in both spatial and frequency domains. Target noise corresponds to the Gaussian noise with a level of 15, which the denoising network has been pretrained to remove.