Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Image Restoration with Mean-Reverting Stochastic Differential Equations

Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön

TL;DR

This paper introduces IR-SDE, a general framework that models image degradation with a mean-reverting forward SDE and learns a reverse-time SDE to restore high-quality images without task-specific priors. A closed-form forward solution enables exact ground-truth score computation, while a maximum-likelihood objective stabilizes training and improves restoration quality. The method is shown to be effective across multiple restoration tasks, achieving competitive results on deraining, deblurring, and denoising and qualitative success in super-resolution, inpainting, and dehazing, including state-of-the-art performance on two rain datasets. Overall, IR-SDE offers a flexible, score-based restoration approach with potential for broad applicability and further efficiency gains.

Abstract

This paper presents a stochastic differential equation (SDE) approach for general-purpose image restoration. The key construction consists in a mean-reverting SDE that transforms a high-quality image into a degraded counterpart as a mean state with fixed Gaussian noise. Then, by simulating the corresponding reverse-time SDE, we are able to restore the origin of the low-quality image without relying on any task-specific prior knowledge. Crucially, the proposed mean-reverting SDE has a closed-form solution, allowing us to compute the ground truth time-dependent score and learn it with a neural network. Moreover, we propose a maximum likelihood objective to learn an optimal reverse trajectory that stabilizes the training and improves the restoration results. The experiments show that our proposed method achieves highly competitive performance in quantitative comparisons on image deraining, deblurring, and denoising, setting a new state-of-the-art on two deraining datasets. Finally, the general applicability of our approach is further demonstrated via qualitative results on image super-resolution, inpainting, and dehazing. Code is available at https://github.com/Algolzw/image-restoration-sde.

Image Restoration with Mean-Reverting Stochastic Differential Equations

TL;DR

This paper introduces IR-SDE, a general framework that models image degradation with a mean-reverting forward SDE and learns a reverse-time SDE to restore high-quality images without task-specific priors. A closed-form forward solution enables exact ground-truth score computation, while a maximum-likelihood objective stabilizes training and improves restoration quality. The method is shown to be effective across multiple restoration tasks, achieving competitive results on deraining, deblurring, and denoising and qualitative success in super-resolution, inpainting, and dehazing, including state-of-the-art performance on two rain datasets. Overall, IR-SDE offers a flexible, score-based restoration approach with potential for broad applicability and further efficiency gains.

Abstract

This paper presents a stochastic differential equation (SDE) approach for general-purpose image restoration. The key construction consists in a mean-reverting SDE that transforms a high-quality image into a degraded counterpart as a mean state with fixed Gaussian noise. Then, by simulating the corresponding reverse-time SDE, we are able to restore the origin of the low-quality image without relying on any task-specific prior knowledge. Crucially, the proposed mean-reverting SDE has a closed-form solution, allowing us to compute the ground truth time-dependent score and learn it with a neural network. Moreover, we propose a maximum likelihood objective to learn an optimal reverse trajectory that stabilizes the training and improves the restoration results. The experiments show that our proposed method achieves highly competitive performance in quantitative comparisons on image deraining, deblurring, and denoising, setting a new state-of-the-art on two deraining datasets. Finally, the general applicability of our approach is further demonstrated via qualitative results on image super-resolution, inpainting, and dehazing. Code is available at https://github.com/Algolzw/image-restoration-sde.
Paper Structure (23 sections, 2 theorems, 49 equations, 18 figures, 11 tables)

This paper contains 23 sections, 2 theorems, 49 equations, 18 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Method
  4. Forward SDE for Image Degradation
  5. Reverse-Time SDE for Image Restoration
  6. Maximum Likelihood Learning
  7. Experiments
  8. Image Deraining
  9. Image Deblurring
  10. Gaussian Image Denoising
  11. Qualitative Experiments
  12. Discussion and Analysis
  13. Reverse-Time Restoration Process
  14. Maximum Likelihood Objective
  15. Time-Varying Theta Schedules
  16. ...and 8 more sections

Key Result

Proposition 3.1

Suppose that the SDE coefficients in equ:ou satisfy $\sigma_t^2 \, / \, \theta_t = 2 \, \lambda^2$ for all times $t$. Then, given any starting state ${x}(s)$ at time $s < t$, the solution to the SDE is where $\bar{\theta}_{s:t} \coloneqq \int^t_s \theta_z \mathop{}\!\mathrm{d} z$ is known and the transition kernel $p({x}(t) {\;|\;} {x}(s)) = \mathcal{N}\bigl({x}(t) {\;|\;} m_{s:t}({x}(s)), v_{s:t

Figures (18)

  • Figure 1: An overview of our proposed construction, where a mean-reverting SDE (\ref{['equ:ou']}) is used for image restoration. The SDE models the degradation process from a high-quality image $x(0)$ to its low-quality counterpart $\mu$, by diffusing $x(0)$ towards a noisy version $\mu + \epsilon$ of the low-quality image. By simulating the corresponding reverse-time SDE, high-quality images can then be restored.
  • Figure 2: Visual results of our IR-SDE method and other deraining approaches on the Rain100H dataset.
  • Figure 3: Visual results of our IR-SDE method compared to other deblurring approaches on the GoPro dataset.
  • Figure 4: Visual results of our methods with other denoising approaches. The total timesteps of IR-SDE is fixed to 100, while the Denoising ODE only requires $\textit{22}$ steps to recover the clean image.
  • Figure 5: Visual results of our IR-SDE method with EDSR on the DIV2K validation dataset for super-resolution. The LQ images are bicubicly upsampled to have the same size as GT images.
  • ...and 13 more figures

Theorems & Definitions (4)

  • Proposition 3.1
  • Proposition 3.2
  • proof
  • proof