Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Elucidating the Design Space of Arbitrary-Noise-Based Diffusion Models

Xingyu Qiu, Mengying Yang, Xinghua Ma, Dong Liang, Fanding Li, Gongning Luo, Wei Wang, Kuanquan Wang, Shuo Li

TL;DR

Theoretically, EDA expands noise pattern flexibility while preserving EDM's modularity, with rigorous proof that increased noise complexity introduces no additional computational overhead during restoration.

Abstract

Although EDM aims to unify the design space of diffusion models, its reliance on fixed Gaussian noise prevents it from explaining emerging flow-based methods that diffuse arbitrary noise. Moreover, our study reveals that EDM's forcible injection of Gaussian noise has adverse effects on image restoration task, as it corrupts the degraded images, overextends the restoration distance, and increases the task's complexity. To interpret diverse methods for handling distinct noise patterns within a unified theoretical framework and to minimize the restoration distance, we propose EDA, which Elucidates the Design space of Arbitrary-noise diffusion models. Theoretically, EDA expands noise pattern flexibility while preserving EDM's modularity, with rigorous proof that increased noise complexity introduces no additional computational overhead during restoration. EDA is validated on three representative medical image denoising and natural image restoration tasks: MRI bias field correction (global smooth noise), CT metal artifact removal (global sharp noise) and natural image shadow removal (local boundary-aware noise). With only 5 sampling steps, competitive results against specialized methods across medical and natural tasks demonstrate EDA's strong generalization capability for image restoration. Code is available at: https://github.com/PerceptionComputingLab/EDA.

Elucidating the Design Space of Arbitrary-Noise-Based Diffusion Models

TL;DR

Theoretically, EDA expands noise pattern flexibility while preserving EDM's modularity, with rigorous proof that increased noise complexity introduces no additional computational overhead during restoration.

Abstract

Although EDM aims to unify the design space of diffusion models, its reliance on fixed Gaussian noise prevents it from explaining emerging flow-based methods that diffuse arbitrary noise. Moreover, our study reveals that EDM's forcible injection of Gaussian noise has adverse effects on image restoration task, as it corrupts the degraded images, overextends the restoration distance, and increases the task's complexity. To interpret diverse methods for handling distinct noise patterns within a unified theoretical framework and to minimize the restoration distance, we propose EDA, which Elucidates the Design space of Arbitrary-noise diffusion models. Theoretically, EDA expands noise pattern flexibility while preserving EDM's modularity, with rigorous proof that increased noise complexity introduces no additional computational overhead during restoration. EDA is validated on three representative medical image denoising and natural image restoration tasks: MRI bias field correction (global smooth noise), CT metal artifact removal (global sharp noise) and natural image shadow removal (local boundary-aware noise). With only 5 sampling steps, competitive results against specialized methods across medical and natural tasks demonstrate EDA's strong generalization capability for image restoration. Code is available at: https://github.com/PerceptionComputingLab/EDA.

Paper Structure

This paper contains 22 sections, 4 theorems, 87 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. EDM can Only Diffuse Gaussian Noise
  4. Diffusion-based Methods for Any Noise Diffusion
  5. Methods
  6. Generalized Forward Process
  7. Deterministic Sampling via PFODE
  8. Experiments
  9. Tasks and Datasets
  10. Experimental Setup
  11. Experimental Results
  12. Conclusion
  13. Acknowledgements
  14. Derivation of $f(\cdot)$, $g(\cdot)$ and $\phi(\cdot)$ in SDE
  15. Derivation of Probability Flow ODE for EDA
  16. ...and 7 more sections

Key Result

Proposition 1

EDA Supports Diffusion and Removal of Arbitrary Noise.

Figures (4)

  • Figure 1: Our EDA supports arbitrary noise patterns diffusion ($\text{N}_\text{Img}$) and enables the reverse process to be initiated directly from the known degraded image. This avoids the extra Gaussian noise corruption ($\text{N}_\text{Gaus}$; top row) of EDM-based methods, and shortens the image restoration distance, while reducing task complexity and achieving high-quality results with fewer sampling steps.
  • Figure 2: (a) EDA extends EDM by enabling any diffused noise patterns while retaining the flexibility of structural parameters, such as noise schedules and training objectives. (b) EDA enables arbitrary noise diffusion avoiding extra Gaussian noise, and reducing the image restoration distance during the reverse process.
  • Figure 3: Our EDA produces the closest results to the ground truth and has competitive image restoration performance. The enlarged view of the green box in the bias field correction is the bias field.
  • Figure 4: Our EDA samples in less than $5$ steps achieve or even surpass the Refusion sampling in $100$ steps.

Theorems & Definitions (4)

  • Proposition 1
  • Proposition 2
  • Proposition 3
  • Proposition 4