2-Shots in the Dark: Low-Light Denoising with Minimal Data Acquisition

TL;DR

This work tackles the data scarcity problem in low-light denoising by introducing a noise synthesis method that requires only a single noisy image and a single dark frame per ISO. It separates signal-dependent noise (Poisson) from signal-independent noise and uses a Fourier-domain spectral sampling approach with phase randomization and iterative histogram matching to generate diverse, realistic noise realizations. The synthesized data enables training denoisers that achieve competitive, and often state-of-the-art, performance on multiple benchmarks while generalizing across sensors. The approach reduces data acquisition and calibration effort, facilitating practical deployment of learning-based denoisers in real-world low-light photography scenarios.

Abstract

Raw images taken in low-light conditions are very noisy due to low photon count and sensor noise. Learning-based denoisers have the potential to reconstruct high-quality images. For training, however, these denoisers require large paired datasets of clean and noisy images, which are difficult to collect. Noise synthesis is an alternative to large-scale data acquisition: given a clean image, we can synthesize a realistic noisy counterpart. In this work, we propose a general and practical noise synthesis method that requires only one single noisy image and one single dark frame per ISO setting. We represent signal-dependent noise with a Poisson distribution and introduce a Fourier-domain spectral sampling algorithm to accurately model signal-independent noise. The latter generates diverse noise realizations that maintain the spatial and statistical properties of real sensor noise. As opposed to competing approaches, our method neither relies on simplified parametric models nor on large sets of clean-noisy image pairs. Our synthesis method is not only accurate and practical, it also leads to state-of-the-art performances on multiple low-light denoising benchmarks.

Figures (21)

• Figure 1: Different from classic denoising pipelines that require large amounts of paired data, we propose a precise sensor noise synthesis method that requires only a single noisy image and a single dark frame. From these two inputs, our algorithm accurately reproduces the sensor noise distribution without the need for complex parameter calibration.
• Figure 2: Overview of the spectral sampling algorithm. A Gaussian blur is applied to the real dark frame $I_{\text{dark}}$ to estimate the fixed-pattern component $S$, which is then subtracted to obtain the stochastic residual $R$. In the Fourier domain, we retain the magnitude $|\widehat{R}|$ and perform phase randomization using a uniform random phase $\xi$ to obtain a new noise realization $N^{(0)}$. Subsequently, $K$ iterations of histogram matching and spectral correction are applied to preserve both the marginal distribution and the spectral characteristics of the noise.
• Figure 3: Result comparison of denoisers trained on data synthesized using different methods, along with a comparison to a denoiser trained on real pairs from the SID training set. The first example is from the SID test set, the second from the ELD test set. For both examples, our method produces cleaner results with fewer artifacts than other methods. Best viewed zoomed in. More results are provided in the appendix.
• Figure 4: Effect of inter-channel correlation (ICC). First row: Noise samples (ISO=50) from different methods, with row-wise averaged correlation matrices for Bayer channels (R, Gr, Gb, B). Second row: A clean-noisy pair and denoising results when denoisers are trained on different with different noise samplers. Real sensor noise exhibits cross-channel correlations, but ELD and our method without correlation enforcement produce nearly independent channels (near-diagonal matrices). Only our full method replicates the correlations. Consequently, denoisers trained with ELD or without correlation enforcement show residual banding artifacts, while our full method yields clean results.
• Figure 5: Effect of iterative histogram matching (IHM). Column 1-2: synthetic noise without and with IHM (with histogram comparison and KLD to real noise). Column 3: a real clean-noisy pair. Column 4-5: the denoising results when training with data synthesized without and with IHM. Without IHM, there is a noticeable distribution misalignment between the synthetic and real noise, and the denoising result exhibits color artifacts. Whereas IHM improves noise realism and restoration quality. Best viewed zoomed in.