Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge

Conghan Yue, Zhengwei Peng, Junlong Ma, Shiyan Du, Pengxu Wei, Dongyu Zhang

TL;DR

The paper addresses restoring HQ images from degraded LQ inputs in ill-posed inverse problems. It introduces the Generalized Ornstein-Uhlenbeck Bridge (GOUB), a diffusion-bridge framework built on a GOU process and Doob's $h$-transform to realize direct point-to-point HQ↔LQ mappings, plus a Mean-ODE variant for efficient, detail-preserving restoration. By deriving closed-form forward/backward transitions and a maximum-likelihood training objective, the authors show GOUB subsumes several diffusion-bridge models as special cases and achieve state-of-the-art results across inpainting, deraining, and super-resolution. Empirical results, ablations, and theoretical analysis demonstrate GOUB's universality, efficiency, and practical impact for robust image restoration, accompanied by public code for reproducibility.

Abstract

Diffusion models exhibit powerful generative capabilities enabling noise mapping to data via reverse stochastic differential equations. However, in image restoration, the focus is on the mapping relationship from low-quality to high-quality images. Regarding this issue, we introduce the Generalized Ornstein-Uhlenbeck Bridge (GOUB) model. By leveraging the natural mean-reverting property of the generalized OU process and further eliminating the variance of its steady-state distribution through the Doob's h-transform, we achieve diffusion mappings from point to point enabling the recovery of high-quality images from low-quality ones. Moreover, we unravel the fundamental mathematical essence shared by various bridge models, all of which are special instances of GOUB and empirically demonstrate the optimality of our proposed models. Additionally, we present the corresponding Mean-ODE model adept at capturing both pixel-level details and structural perceptions. Experimental outcomes showcase the state-of-the-art performance achieved by both models across diverse tasks, including inpainting, deraining, and super-resolution. Code is available at \url{https://github.com/Hammour-steak/GOUB}.

Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge

TL;DR

The paper addresses restoring HQ images from degraded LQ inputs in ill-posed inverse problems. It introduces the Generalized Ornstein-Uhlenbeck Bridge (GOUB), a diffusion-bridge framework built on a GOU process and Doob's -transform to realize direct point-to-point HQ↔LQ mappings, plus a Mean-ODE variant for efficient, detail-preserving restoration. By deriving closed-form forward/backward transitions and a maximum-likelihood training objective, the authors show GOUB subsumes several diffusion-bridge models as special cases and achieve state-of-the-art results across inpainting, deraining, and super-resolution. Empirical results, ablations, and theoretical analysis demonstrate GOUB's universality, efficiency, and practical impact for robust image restoration, accompanied by public code for reproducibility.

Abstract

Diffusion models exhibit powerful generative capabilities enabling noise mapping to data via reverse stochastic differential equations. However, in image restoration, the focus is on the mapping relationship from low-quality to high-quality images. Regarding this issue, we introduce the Generalized Ornstein-Uhlenbeck Bridge (GOUB) model. By leveraging the natural mean-reverting property of the generalized OU process and further eliminating the variance of its steady-state distribution through the Doob's h-transform, we achieve diffusion mappings from point to point enabling the recovery of high-quality images from low-quality ones. Moreover, we unravel the fundamental mathematical essence shared by various bridge models, all of which are special instances of GOUB and empirically demonstrate the optimality of our proposed models. Additionally, we present the corresponding Mean-ODE model adept at capturing both pixel-level details and structural perceptions. Experimental outcomes showcase the state-of-the-art performance achieved by both models across diverse tasks, including inpainting, deraining, and super-resolution. Code is available at \url{https://github.com/Hammour-steak/GOUB}.
Paper Structure (30 sections, 7 theorems, 66 equations, 8 figures, 8 tables)

This paper contains 30 sections, 7 theorems, 66 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Score-based Diffusion Model
  4. Generalized Ornstein-Uhlenbeck process
  5. Doob's h-transform
  6. GOUB
  7. Forward and backward process
  8. Training object
  9. Mean-ODE
  10. Experiments
  11. Image Inpainting.
  12. Image Deraining.
  13. Image Super-Resolution.
  14. Superiority of Mean-ODE.
  15. Analysis
  16. ...and 15 more sections

Key Result

Proposition 3.1

Let $\mathbf x_t$ be a finite random variable describing by the given generalized Ornstein-Uhlenbeck process eq4, suppose $\mathbf{x}_T=\bm\mu$, the evolution of its marginal distribution $p(\mathbf{x}_t\mid \mathbf{x}_T)$ satisfies the following SDE: Additionally, the forward transition $p(\mathbf{x}_t\mid \mathbf{x}_0, \mathbf{x}_T)$ is given by:

Figures (8)

  • Figure 1: Overview of the proposed GOUB for image restoration. The GOU process is capable of transferring an HQ image into a noisy LQ image. Additionally, through the application of h-transform, we can eliminate the noise on LQ, enabling the GOUB model to precisely bridge the gap between HQ and LQ.
  • Figure 2: Qualitative comparison of the visual results of different inpainting methods on the CelebA-HQ dataset with thin mask.
  • Figure 3: Qualitative comparison of the visual results of different deraining methods on the Rain100H dataset.
  • Figure 4: Qualitative comparison of the visual results of different 4x super-resolution methods on the DIV2K dataset.
  • Figure 5: Qualitative comparison with the different bridge models in many tasks.
  • ...and 3 more figures

Theorems & Definitions (7)

  • Proposition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Proposition 5.1
  • Proposition 2.1
  • Theorem 3.1
  • Theorem 4.1