Super-Resolution with Structured Motion

TL;DR

This paper investigates pushing super-resolution beyond small gains by exploiting structured motion and sparse priors within a box-deconvolution framework. The forward model links high-resolution content $J$ to low-resolution measurements via $I = (J \otimes Q \otimes B) \downarrow_f$, and interlacing multiple subpixel captures yields $H = J \otimes B$, enabling deconvolution with a box despite its noninvertibility. It demonstrates that sparsity-promoting priors, notably total variation, enable near-perfect reconstructions through convex optimization, and reveals that controlled motion blur can provide sub-pixel information from blurred measurements, sometimes with a single exposure. The authors validate the approach through simulations and real experiments using a camera on a computer-controlled stage, achieving SR factors up to $f=8$ and illustrating the potential for very high-resolution imaging in constrained optical setups.

Abstract

We consider the limits of super-resolution using imaging constraints. Due to various theoretical and practical limitations, reconstruction-based methods have been largely restricted to small increases in resolution. In addition, motion-blur is usually seen as a nuisance that impedes super-resolution. We show that by using high-precision motion information, sparse image priors, and convex optimization, it is possible to increase resolution by large factors. A key operation in super-resolution is deconvolution with a box. In general, convolution with a box is not invertible. However, we obtain perfect reconstructions of sparse signals using convex optimization. We also show that motion blur can be helpful for super-resolution. We demonstrate that using pseudo-random motion it is possible to reconstruct a high-resolution target using a single low-resolution image. We present numerical experiments with simulated data and results with real data captured by a camera mounted on a computer controlled stage.

Figures (14)

• Figure 1: Using motion blur for super-resolution.
• Figure 2: Interlacing $f \times f$ low-resolution images taken in a grid of sub-pixel locations, we obtain the convolution of the high-resolution target with a two-dimensional box filter of width $f$.
• Figure 3: (a) One-dimensional box and Gaussian (top) and their Fourier transforms (bottom). The red plots correspond to a Gaussian and the blue plots correspond to a box. (b) Bode plot (magnitude of Fourier transform in dB) of a two-dimensional discrete box filter of width 4.
• Figure 4: Super-resolution with $f=8$. By interlacing 64 low-resolution images, we obtain the measurements $H = J \otimes B$. Deconvolving $H$ with a TV prior leads to an almost perfect reconstruction.
• Figure 5: Deconvolution of $H$ with a quadratic smoothness prior leads to an overly smooth image and ringing artifacts while using a TV prior yields an almost perfect reconstruction.