Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Residual Diffusion Bridge Model for Image Restoration

Hebaixu Wang, Jing Zhang, Haoyang Chen, Haonan Guo, Di Wang, Jiayi Ma, Bo Du

TL;DR

RDBM addresses universal image restoration by introducing a residual-guided diffusion bridge framework that adaptively perturbs noise in degraded regions while preserving intact areas. It reformulates the forward and reverse diffusion bridge SDEs with residuals and Doob’s $h$-transform, and proves that existing bridge models are special cases within this unified formulation. The method derives closed-form marginals, defines a residual-to-noise ratio $R(t)$ to ensure smooth, region-aware transitions, and uses a neural predictor to realize the residual-perturbed reverse path. Extensive experiments across five restoration tasks show state-of-the-art performance and strong generalization, with code publicly available.

Abstract

Diffusion bridge models establish probabilistic paths between arbitrary paired distributions and exhibit great potential for universal image restoration. Most existing methods merely treat them as simple variants of stochastic interpolants, lacking a unified analytical perspective. Besides, they indiscriminately reconstruct images through global noise injection and removal, inevitably distorting undegraded regions due to imperfect reconstruction. To address these challenges, we propose the Residual Diffusion Bridge Model (RDBM). Specifically, we theoretically reformulate the stochastic differential equations of generalized diffusion bridge and derive the analytical formulas of its forward and reverse processes. Crucially, we leverage the residuals from given distributions to modulate the noise injection and removal, enabling adaptive restoration of degraded regions while preserving intact others. Moreover, we unravel the fundamental mathematical essence of existing bridge models, all of which are special cases of RDBM and empirically demonstrate the optimality of our proposed models. Extensive experiments are conducted to demonstrate the state-of-the-art performance of our method both qualitatively and quantitatively across diverse image restoration tasks. Code is publicly available at https://github.com/MiliLab/RDBM.

Residual Diffusion Bridge Model for Image Restoration

TL;DR

RDBM addresses universal image restoration by introducing a residual-guided diffusion bridge framework that adaptively perturbs noise in degraded regions while preserving intact areas. It reformulates the forward and reverse diffusion bridge SDEs with residuals and Doob’s -transform, and proves that existing bridge models are special cases within this unified formulation. The method derives closed-form marginals, defines a residual-to-noise ratio to ensure smooth, region-aware transitions, and uses a neural predictor to realize the residual-perturbed reverse path. Extensive experiments across five restoration tasks show state-of-the-art performance and strong generalization, with code publicly available.

Abstract

Diffusion bridge models establish probabilistic paths between arbitrary paired distributions and exhibit great potential for universal image restoration. Most existing methods merely treat them as simple variants of stochastic interpolants, lacking a unified analytical perspective. Besides, they indiscriminately reconstruct images through global noise injection and removal, inevitably distorting undegraded regions due to imperfect reconstruction. To address these challenges, we propose the Residual Diffusion Bridge Model (RDBM). Specifically, we theoretically reformulate the stochastic differential equations of generalized diffusion bridge and derive the analytical formulas of its forward and reverse processes. Crucially, we leverage the residuals from given distributions to modulate the noise injection and removal, enabling adaptive restoration of degraded regions while preserving intact others. Moreover, we unravel the fundamental mathematical essence of existing bridge models, all of which are special cases of RDBM and empirically demonstrate the optimality of our proposed models. Extensive experiments are conducted to demonstrate the state-of-the-art performance of our method both qualitatively and quantitatively across diverse image restoration tasks. Code is publicly available at https://github.com/MiliLab/RDBM.
Paper Structure (37 sections, 5 theorems, 92 equations, 20 figures, 11 tables, 2 algorithms)

This paper contains 37 sections, 5 theorems, 92 equations, 20 figures, 11 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Diffusion Bridge Models
  5. Ornstein Uhlenbeck Process
  6. Residual Diffusion Bridge Models
  7. Generalized Forward Process
  8. Reverse Process and Training Objective
  9. Analysis
  10. Experiments
  11. Datasets and Evaluation Metrics
  12. Implementation Details
  13. Comparative Experiments
  14. Ablation Study
  15. Zero-Short Real-world Generation
  16. ...and 22 more sections

Key Result

Proposition 1

Let $\mathbf{x}_t$ be a finite random variable governed by the generalized diffusion bridge process in Eq. sec:method_eq1, with terminal condition $\mathbf{x}_T = \boldsymbol{\mu}$. The evolution of its marginal distribution $p(\mathbf{x}_t \mid \mathbf{x}_T)$ satisfies the following SDE under a fix where $\overline{\theta}_{s:t} = \int_s^t \theta_z dz$ and $\boldsymbol{\pi}\in \mathbb{R}$. See Su

Figures (20)

  • Figure 1: Typical diffusion processes for image restoration. (a) Standard diffusion maps high-quality images to the Gaussian noise domain. (b) Mean-reverting diffusion drives terminal state toward a low-quality domain with stationary noise. (c) Diffusion bridge establishes direct probabilistic transitions between known distributions. All inject noise globally, disrupting overall structures and constraining transitions. (d) In contrast, our RDBM selectively reconstructs degraded regions (e.g., the doll) while preserving intact areas (e.g., the background), thus avoiding redundant restoration.
  • Figure 2: A schematic of Residual Diffusion Bridge Models. RDBM utilizes a diffusion process guided by Doob’s $h$-transform towards an endpoint $\mathbf{x}_T = \boldsymbol{\mu}$ free from stationary noise $\lambda \epsilon$. Modulated by the residual component $\boldsymbol{\pi} = \mathbf{x}_0 - \mathbf{x}_T$, the noise perturbation is selectively imposed on different regions with diverse degradation levels, thereby constructing probabilistic paths. Besides, it learns to reverse the process by matching the residual bridge score functions, facilitating an adaptive inversion from $\mathbf{x}_T\sim p_{prior}(\boldsymbol{x})$ to $\mathbf{x}_0 \sim p_{data}(\boldsymbol{x})$.
  • Figure 3: Overview of mainstream diffusion processes via SDEs, all of which are special cases of our framework. (a) OU process maps the data distribution to the prior distribution with noise. (b) OU bridge constructs probabilistic transition paths between given distributions. (c) Brownian bridge models linear expectations of intermediate states. (d) Our RDBM leverages residuals from paired distributions to adaptively modify the transition trajectories, maintaining a smooth residual-to-noise ratio.
  • Figure 4: Visualization comparison with state-of-the-art methods on deraining. Zoom in for best view.
  • Figure 5: Visualization results of different NFEs in a blurry night-time scene. Zoom in for best view.
  • ...and 15 more figures

Theorems & Definitions (5)

  • Proposition 1
  • Proposition 2
  • Theorem 1
  • Theorem 2
  • Proposition 3