Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trustworthy Image Super-Resolution via Generative Pseudoinverse

Andreas Floros, Seyed-Mohsen Moosavi-Dezfooli, Pier Luigi Dragotti

TL;DR

The paper tackles trustworthy image super-resolution by enforcing strict consistency with the degradation model $\mathcal{D}$ and the measurements $\mathbf{y}=\mathcal{D}(\mathbf{x})$. It introduces Flow-Based Generative Pseudoinverse to learn a bijection $\mathcal{X}\leftrightarrow\mathcal{Y}\times\mathcal{Z}$, with $\mathcal{Z}$ encoding a generalized kernel, and then refines this kernel with a latent DDPM operating in $\mathcal{Z}$ to achieve asymptotically consistent posterior samples $p(\mathbf{x}|\mathbf{y})$. Experiments on 16$\times$16 to 128$\times$128 face SR (FFHQ/CelebA-HQ) show improved PSNR and SSIM and lower measurement-consistency error relative to strong baselines, especially at low neural evaluation budgets, with further gains in consistency at higher budgets. The approach highlights a principled balance between fidelity and hallucination suppression and suggests broad applicability to other conditioned inverse problems beyond SR.

Abstract

We consider the problem of trustworthy image restoration, taking the form of a constrained optimization over the prior density. To this end, we develop generative models for the task of image super-resolution that respect the degradation process and that can be made asymptotically consistent with the low-resolution measurements, outperforming existing methods by a large margin in that respect.

Trustworthy Image Super-Resolution via Generative Pseudoinverse

TL;DR

The paper tackles trustworthy image super-resolution by enforcing strict consistency with the degradation model and the measurements . It introduces Flow-Based Generative Pseudoinverse to learn a bijection , with encoding a generalized kernel, and then refines this kernel with a latent DDPM operating in to achieve asymptotically consistent posterior samples . Experiments on 1616 to 128128 face SR (FFHQ/CelebA-HQ) show improved PSNR and SSIM and lower measurement-consistency error relative to strong baselines, especially at low neural evaluation budgets, with further gains in consistency at higher budgets. The approach highlights a principled balance between fidelity and hallucination suppression and suggests broad applicability to other conditioned inverse problems beyond SR.

Abstract

We consider the problem of trustworthy image restoration, taking the form of a constrained optimization over the prior density. To this end, we develop generative models for the task of image super-resolution that respect the degradation process and that can be made asymptotically consistent with the low-resolution measurements, outperforming existing methods by a large margin in that respect.
Paper Structure (9 sections, 3 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 9 sections, 3 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Normalizing Flows
  3. Diffusion Models
  4. Proposed Method
  5. Flow-Based Generative Pseudoinverse
  6. Refinement of the Kernel via Diffusion
  7. Experiments
  8. Results
  9. Discussion

Figures (3)

  • Figure 1: Linear SR problem in $\mathbb{R}^2$. We have $\mathbf{x} \sim\mathcal{N}(\mathbf{0} ,\mathbf{I} )$ and $p(\mathrm{y}|\mathbf{x} )=\delta_{\mathcal{D}\mathbf{x} -\mathrm{y}}$. The intersection of the planes $\mathrm{z}=\mathcal{D}(\mathbf{x} )-\mathrm{y}$ and $\mathrm{z}=0$ defines the feasibility set. The marked points outside of this set correspond to solutions that optimize the prior at the cost of consistency. We deem these to not be trustworthy as they violate the constraints of the problem.
  • Figure 2: Results of our $16\times16\to128\times128$ SR.
  • Figure 3: $8\times$ SR. Based on the restored eyes, nose and lips, our method surpasses the competitors.

Theorems & Definitions (6)

  • Definition 1: Consistency
  • Definition 2: Generalized Inverse
  • Remark 1
  • Definition 3: Generalized Kernel
  • Remark 2
  • Remark 3