Deep Variational Network Toward Blind Image Restoration

TL;DR

This work tackles blind image restoration by modeling the degradation process with a Bayesian generative model that allows pixel-wise non-i.i.d. Gaussian noise and, for super-resolution, an anisotropic Gaussian kernel. It introduces an amortized stochastic variational inference scheme in which the posterior over latent clean images, per-pixel noise variances, and degradation kernels is factorized and parameterized by neural networks (SNet, KNet, and RNet), enabling a unified framework that jointly estimates degradation and restores images. The approach yields VIRNet, which achieves state-of-the-art or competitive results on both denoising and blind SR tasks, while maintaining favorable efficiency and providing interpretable degradation estimates through the learned variance maps and kernel parameters. This probabilistic, end-to-end framework offers a principled path toward robust IR in real-world settings with complex noise and unknown degradations, with potential for extension to more general kernel families and degradation models.

Abstract

Blind image restoration (IR) is a common yet challenging problem in computer vision. Classical model-based methods and recent deep learning (DL)-based methods represent two different methodologies for this problem, each with their own merits and drawbacks. In this paper, we propose a novel blind image restoration method, aiming to integrate both the advantages of them. Specifically, we construct a general Bayesian generative model for the blind IR, which explicitly depicts the degradation process. In this proposed model, a pixel-wise non-i.i.d. Gaussian distribution is employed to fit the image noise. It is with more flexibility than the simple i.i.d. Gaussian or Laplacian distributions as adopted in most of conventional methods, so as to handle more complicated noise types contained in the image degradation. To solve the model, we design a variational inference algorithm where all the expected posteriori distributions are parameterized as deep neural networks to increase their model capability. Notably, such an inference algorithm induces a unified framework to jointly deal with the tasks of degradation estimation and image restoration. Further, the degradation information estimated in the former task is utilized to guide the latter IR process. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.

Figures (11)

• Figure 1: Illustration of the graphical model of the proposed method.
• Figure 2: The inference framework of the proposed generative model for blind image super-resolution. It can be decomposed into two sub-tasks of degradation estimation and image restoration. Given any corrupted image $\bm{y}$, we firstly infer the posteriori parameters $\bm{\beta}$ of $q(\bm{\sigma}^2|\bm{y})$ by SNet and $\{m,\eta_1, \eta_2\}$ of $q(\bm{\varLambda}|\bm{y})$ by KNet in the phase of degradation estimation, and then recover the desirable high-quality image (i.e., the mean value $\bm{\mu}$ of $q(\bm{z}|\bm{y},\bm{\sigma}^2,\bm{\varLambda})$) by RNet under the guidance of the estimated degradation information.
• Figure 3: (a) The spatially variant map $\bm{M}$ for noise generation in training data. (b1)-(d1): Three different $\bm{M}$s on testing data in Cases 1-3. (b2)-(d2): Predicted $\bm{M}$s by our method on testing data.
• Figure 4: Denoising results of different competing methods on three typical test examples of synthetic non-i.i.d. Gaussian Noise Removal.
• Figure 5: Denoising results of all competing methods on two typical real-world examples from DND Plotz2017 (upper) and SIDD Abdelhamed2018 (lower) datasets.