Blind Super-Resolution via Meta-learning and Markov Chain Monte Carlo Simulation

TL;DR

This work tackles blind single image super-resolution when the degradation kernel is unknown. It introduces MLMC, a two-phase, unsupervised framework that learns kernel priors from organized randomness via Markov Chain Monte Carlo (MCMC) simulation and refines them with a meta-learning-based alternating optimization, incorporating network-level Langevin dynamics for convergence. The MCKA phase provides a plug-and-play, data-free kernel prior by sampling from random Gaussian kernels and updating a lightweight kernel generator, while the MLAO phase adaptively updates both kernel and HR image estimators to achieve robust restoration. Empirically, MLMC demonstrates superior generalization to out-of-distribution and motion kernels, resilience to noise, and competitive efficiency compared with both unsupervised and supervised state-of-the-art methods, highlighting its practical potential for real-world blind SR without heavy training requirements.

Abstract

Learning-based approaches have witnessed great successes in blind single image super-resolution (SISR) tasks, however, handcrafted kernel priors and learning based kernel priors are typically required. In this paper, we propose a Meta-learning and Markov Chain Monte Carlo (MCMC) based SISR approach to learn kernel priors from organized randomness. In concrete, a lightweight network is adopted as kernel generator, and is optimized via learning from the MCMC simulation on random Gaussian distributions. This procedure provides an approximation for the rational blur kernel, and introduces a network-level Langevin dynamics into SISR optimization processes, which contributes to preventing bad local optimal solutions for kernel estimation. Meanwhile, a meta-learning-based alternating optimization procedure is proposed to optimize the kernel generator and image restorer, respectively. In contrast to the conventional alternating minimization strategy, a meta-learning-based framework is applied to learn an adaptive optimization strategy, which is less-greedy and results in better convergence performance. These two procedures are iteratively processed in a plug-and-play fashion, for the first time, realizing a learning-based but plug-and-play blind SISR solution in unsupervised inference. Extensive simulations demonstrate the superior performance and generalization ability of the proposed approach when comparing with state-of-the-arts on synthesis and real-world datasets. The code is available at https://github.com/XYLGroup/MLMC.

Figures (9)

• Figure 1: The overall framework of the proposed MLMC.
• Figure 2: The overview of the Markov Chain Monte Carlo kernel approximation (MCKA). ($\bm{a}$) The approximated blur kernel $\bm{k}^l=\text{G}_{\bm{k}}(\bm{z}_{\bm{k}},\bm{\phi}^l_{\bm{k},\textit{MC}})$ is generated from the given noise $\bm{z}_{\bm{k}}$ with a fully-connected network (FCN) $\text{G}_{\bm{k}}$, the parameters of which are iteratively updated, thereof resulting in Markov chain with possible transition$p(\bm{\phi}^{l+1}_{\bm{k},\textit{MC}}|\bm{\phi}^{l}_{\bm{k},\textit{MC}},\bm{z}_{\bm{k}},\bm{x},\bm{y})$. ($\bm{b}$) Left: the network parameters (blue crosses) are optimized by gradient descent-based algorithm iteratively with respect to the MCMC loss function in Eq. (\ref{['eq:MCMC loss function']}). The obtained approximated kernels present a trend of non-monotonically decreasing MCMC loss. Right: the three panels show the trajectory of optimization on $\bm{\phi}^{l}_{\bm{k},\textit{MC}}$ over the geometry of MCMC loss (blue crosses). From an initial model (obtained from the last MLAO phase), the parameters $\bm{\phi}^{l}_{\bm{k},\textit{MC}}$ are updated via gradient descent-based algorithm. ($\bm{c}$) Left: the MCMC simulations sample kernels (green crosses) from random Gaussian distributions with respect to a posterior that the sampled kernel should minimize the MCMC loss as well as close to the last generated sample. What stands out in this chart is the fluctuated convergence of the sampled kernel PSNR. This is caused by the posterior distribution formed by the Markov chain possible transition $p(\bm{\phi}^{l+1}_{\bm{k},\textit{MC}}|\bm{\phi}^{l}_{\bm{k},\textit{MC}},\bm{z}_{\bm{k}},\bm{x},\bm{y})$. Right: the three panels illustrate that sufficient MCMC sampling will lead to an approximation of posterior distribution underlying the minimized MCMC loss with respect to $\bm{k}^*$. The random samples (green crosses) are distributed on posterior distribution with the guidance of probability density from LR reconstruction loss. Those kernels with less MCMC loss will be sampled with higher probability along with the MCMC process. We note that $\bm{k}^*$ is the generated blur kernel with respect to $\bm{\phi}_{\bm{k}}^*$.
• Figure 3: Blur kernels are randomly sampled by the Monte Carlo simulation from different Gaussian distributions, e.g., different variances ($\bm{\Sigma}$) and rotation angle ($\rho$).
• Figure 4: Image PSNR performance of the proposed MLMC method on Set5 with different hyper-parameter combinations: the number of the Monte Carlo sampling times $T$ and the number of meta-learning optimization intervals $P$.
• Figure 5: Visual results of different methods on public datasets for scale factor 4. Estimated/ground-truth kernels are shown on the top right.