Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion-Based sRGB Real Noise Generation via Prompt-Driven Noise Representation Learning

Jaekyun Ko, Dongjin Kim, Soomin Lee, Guanghui Wang, Tae Hyun Kim

TL;DR

This work proposes a novel framework called Prompt-Driven Noise Generation (PNG), capable of acquiring high-dimensional prompt features that capture the characteristics of real-world input noise and creating a variety of realistic noisy images consistent with the distribution of the input noise.

Abstract

Denoising in the sRGB image space is challenging due to noise variability. Although end-to-end methods perform well, their effectiveness in real-world scenarios is limited by the scarcity of real noisy-clean image pairs, which are expensive and difficult to collect. To address this limitation, several generative methods have been developed to synthesize realistic noisy images from limited data. These generative approaches often rely on camera metadata during both training and testing to synthesize real-world noise. However, the lack of metadata or inconsistencies between devices restricts their usability. Therefore, we propose a novel framework called Prompt-Driven Noise Generation (PNG). This model is capable of acquiring high-dimensional prompt features that capture the characteristics of real-world input noise and creating a variety of realistic noisy images consistent with the distribution of the input noise. By eliminating the dependency on explicit camera metadata, our approach significantly enhances the generalizability and applicability of noise synthesis. Comprehensive experiments reveal that our model effectively produces realistic noisy images and show the successful application of these generated images in removing real-world noise across various benchmark datasets.

Diffusion-Based sRGB Real Noise Generation via Prompt-Driven Noise Representation Learning

TL;DR

This work proposes a novel framework called Prompt-Driven Noise Generation (PNG), capable of acquiring high-dimensional prompt features that capture the characteristics of real-world input noise and creating a variety of realistic noisy images consistent with the distribution of the input noise.

Abstract

Denoising in the sRGB image space is challenging due to noise variability. Although end-to-end methods perform well, their effectiveness in real-world scenarios is limited by the scarcity of real noisy-clean image pairs, which are expensive and difficult to collect. To address this limitation, several generative methods have been developed to synthesize realistic noisy images from limited data. These generative approaches often rely on camera metadata during both training and testing to synthesize real-world noise. However, the lack of metadata or inconsistencies between devices restricts their usability. Therefore, we propose a novel framework called Prompt-Driven Noise Generation (PNG). This model is capable of acquiring high-dimensional prompt features that capture the characteristics of real-world input noise and creating a variety of realistic noisy images consistent with the distribution of the input noise. By eliminating the dependency on explicit camera metadata, our approach significantly enhances the generalizability and applicability of noise synthesis. Comprehensive experiments reveal that our model effectively produces realistic noisy images and show the successful application of these generated images in removing real-world noise across various benchmark datasets.
Paper Structure (33 sections, 16 equations, 9 figures, 14 tables)

This paper contains 33 sections, 16 equations, 9 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed Method
  4. Preliminaries
  5. Overall Flow: PNG
  6. Prompt Autoencoder
  7. Prompt Encoder
  8. Decoder
  9. Prompt DiT (P-DiT)
  10. Experiments
  11. Experimental Setup
  12. Real-World sRGB Noise Generation and Removal
  13. Application: Metadata-Free Noise Generation
  14. Ablation Study
  15. Conclusion
  16. ...and 18 more sections

Figures (9)

  • Figure 1: Noise Generation Comparison. (a) Metadata-dependent approach. (b) Ours.
  • Figure 2: Overview of the proposed method. (a) Training pipeline. (b) Inference pipeline.
  • Figure 3: (a) Sketch of the Prompt Autoencoder (PAE). (b) Details of Global and Local Prompt Blocks.
  • Figure 4: Visualization of synthetic noisy images on the SIDD validation set. From left to right: C2N, NeCA-W, NAFlow, Ours (PNG), and real noisy images.
  • Figure 5: Visual comparison on denoising results with PSNR$\uparrow$ on SIDD validation set from DnCNN trained on each method. For more qualitative results, please refer to the supplementary material.
  • ...and 4 more figures