Spatially-Variant Degradation Model for Dataset-free Super-resolution

TL;DR

This work tackles dataset-free Blind Image Super-Resolution (BISR) by introducing a Spatially-Variant Degradation Model (SVDSR) that assigns a per-pixel degradation kernel. Each pixel's kernel is a linear combination of a small, learnable dictionary of atom degradations, with coefficients computed via fuzzy membership based on local texture of a tentative HR image; the atoms themselves are anisotropic Gaussians with learnable parameters. The authors formulate a Probabilistic BISR framework with a joint likelihood in both spatial and frequency domains, plus image and kernel priors, and solve it with a Monte Carlo EM (MCEM) algorithm that alternates Langevin-based sampling of latent variables and ADAM-based parameter updates. Experiments on synthetic and real data show ~1 dB average improvements over state-of-the-art dataset-free methods, with competitive performance against non-blind baselines and favorable model efficiency, indicating strong practical potential for real-world SR without paired datasets. The approach provides a compact, physically-informed pathway to model spatially varying degradation and demonstrates how fuzzy texture cues can regularize per-pixel kernel inference in a dataset-free setting.

Abstract

This paper focuses on the dataset-free Blind Image Super-Resolution (BISR). Unlike existing dataset-free BISR methods that focus on obtaining a degradation kernel for the entire image, we are the first to explicitly design a spatially-variant degradation model for each pixel. Our method also benefits from having a significantly smaller number of learnable parameters compared to data-driven spatially-variant BISR methods. Concretely, each pixel's degradation kernel is expressed as a linear combination of a learnable dictionary composed of a small number of spatially-variant atom kernels. The coefficient matrices of the atom degradation kernels are derived using membership functions of fuzzy set theory. We construct a novel Probabilistic BISR model with tailored likelihood function and prior terms. Subsequently, we employ the Monte Carlo EM algorithm to infer the degradation kernels for each pixel. Our method achieves a significant improvement over other state-of-the-art BISR methods, with an average improvement of 1 dB (2x).Code will be released at https://github.com/shaojieguoECNU/SVDSR.

Figures (4)

• Figure 1: The framework of the entire model and its solution (MCEM inference algorithm). The proposed Spatially-Variant Degradation Model consists of of a dictionary of learnable atom operators $\{\boldsymbol{D}_i\}_{i=1}^{N_\mathcal{D}}$ and corresponding coefficient matrices $\{\boldsymbol{W}_i\}_{i=1}^{N_\mathcal{D}}$ . Each atom degradation operator is determined by three learnable parameters $\theta_i$, $\sigma_{i,1}$ and $\sigma_{i,2}$. The coefficient matrices $\{\boldsymbol{W}_i\}_{i=1}^{N_\mathcal{D}}$ are obtained from the tentative HR image $\Tilde{\boldsymbol{x}}$. Image prior, kernel prior and likelihood are suggested to solve the proposed spatially-variant degradation model under the MAP framework. In the inference process, the latent variable $\boldsymbol{z}$ is sampled in the E-Step and parameters $\{\boldsymbol{\Gamma}\}_{i=1}^{N_{\mathcal{D}}}$ and weights $\phi$ of G are updated in M-step.
• Figure 2: Visualization of the synthetic images. Each row, from top to bottom, contains super resolved images from Urban100, DIV2K100 and Manga109, under the scale factor of 2, 3 and 4, respectively.
• Figure 3: (a) Visualization of the coefficient matrices and the corresponding learned atom degradation kernels of the image Baby under the scale factor of 2 in Set 5. (b) Visualization of the synthetic images with the spatially-invariant degradation model (top) and spatially-variant degradation model (bottom). (c) Convergence curves.
• Figure 4: Visualization of the RealSRSet under scale factor of 2(top) and 4(bottom).