Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Physics-Informed Noise Models from Dark Frames for Low-Light Raw Image Denoising

Hansen Feng, Lizhi Wang, Yiqi Huang, Yuzhi Wang, Lin Zhu, Hua Huang

TL;DR

The paper tackles the challenge of low-light RAW denoising by addressing the data dependency of noise modeling. It proposes learning the sensor noise model from dark frames using a physics-informed neural proxy (PNNP) that combines physics-guided noise decoupling (PND), a physics-aware proxy model (PPM), and a differentiable distribution loss (DDL). This approach decouples frame-wise and band-wise noise from pixel-wise noise, constrains the proxy with physical priors, and supervises the distribution with differentiable distribution metrics, enabling accurate noise modeling with limited real data. Extensive experiments on SID, ELD, and LRID demonstrate superior denoising performance and robustness, with low computational cost and high practicality. The work also discusses data-quality considerations and metrics limitations, highlighting the benefits of dark-frame–based modeling and potential hybrid strategies that blend real and synthetic data.

Abstract

Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distribution, while learning-based noise modeling impractically depends on paired real data. In this paper, we propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency. Based on this strategy, we introduce an efficient physics-informed noise neural proxy (PNNP) to approximate the real-world sensor noise model. Specifically, we integrate physical priors into neural proxies and introduce three efficient techniques: physics-guided noise decoupling (PND), physics-aware proxy model (PPM), and differentiable distribution loss (DDL). PND decouples the dark frame into different components and handles different levels of noise flexibly, which reduces the complexity of noise modeling. PPM incorporates physical priors to constrain the synthetic noise, which promotes the accuracy of noise modeling. DDL provides explicit and reliable supervision for noise distribution, which promotes the precision of noise modeling. PNNP exhibits powerful potential in characterizing the real noise distribution. Extensive experiments on public datasets demonstrate superior performance in practical low-light raw image denoising. The source code will be publicly available at the project homepage.

Learning Physics-Informed Noise Models from Dark Frames for Low-Light Raw Image Denoising

TL;DR

The paper tackles the challenge of low-light RAW denoising by addressing the data dependency of noise modeling. It proposes learning the sensor noise model from dark frames using a physics-informed neural proxy (PNNP) that combines physics-guided noise decoupling (PND), a physics-aware proxy model (PPM), and a differentiable distribution loss (DDL). This approach decouples frame-wise and band-wise noise from pixel-wise noise, constrains the proxy with physical priors, and supervises the distribution with differentiable distribution metrics, enabling accurate noise modeling with limited real data. Extensive experiments on SID, ELD, and LRID demonstrate superior denoising performance and robustness, with low computational cost and high practicality. The work also discusses data-quality considerations and metrics limitations, highlighting the benefits of dark-frame–based modeling and potential hybrid strategies that blend real and synthetic data.

Abstract

Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distribution, while learning-based noise modeling impractically depends on paired real data. In this paper, we propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency. Based on this strategy, we introduce an efficient physics-informed noise neural proxy (PNNP) to approximate the real-world sensor noise model. Specifically, we integrate physical priors into neural proxies and introduce three efficient techniques: physics-guided noise decoupling (PND), physics-aware proxy model (PPM), and differentiable distribution loss (DDL). PND decouples the dark frame into different components and handles different levels of noise flexibly, which reduces the complexity of noise modeling. PPM incorporates physical priors to constrain the synthetic noise, which promotes the accuracy of noise modeling. DDL provides explicit and reliable supervision for noise distribution, which promotes the precision of noise modeling. PNNP exhibits powerful potential in characterizing the real noise distribution. Extensive experiments on public datasets demonstrate superior performance in practical low-light raw image denoising. The source code will be publicly available at the project homepage.
Paper Structure (25 sections, 12 equations, 13 figures, 5 tables)

This paper contains 25 sections, 12 equations, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Low-light Raw Image Denoising
  4. Noise Modeling
  5. Method
  6. Strategy and Framework
  7. Physics-guided Noise Decoupling
  8. Physics-aware Proxy Model
  9. Differentiable Distribution Loss
  10. Experiment
  11. Experimental Setting
  12. Implementation details
  13. Denoising Results
  14. ELD dataset and SID dataset
  15. LRID dataset
  16. ...and 10 more sections

Figures (13)

  • Figure 1: The overview of our physics-informed noise neural proxy (PNNP) framework. We have visualized the noise as sRGB images for viewing.
  • Figure 2: The analysis of PND process. The first and second columns plot the noise and its corresponding noise distribution, respectively. "std" refers to the standard deviation of the noise, which characterizes the intensity of the noise. The numerical range is adjusted for the best viewing. Red boxes highlight zoomed-in regions for detailed observation.
  • Figure 3: The overview of our PPM. "nf" represents the number of filters. All 1$\times$1 convolutions in PPM have 16 channels.
  • Figure 4: The principle of DDL. (a) Cumulative distribution function plot, used to observe and measure the distribution distance between the target and the output. (b) Detailed view of CDF, used to illustrate the linear interpolation operation during the query process.
  • Figure 5: Raw image denoising results on images from the ELD dataset. The red color indicates the best results and the blue color indicates the second-best results. (Best viewed with zoom-in)
  • ...and 8 more figures