Video Deblurring by Sharpness Prior Detection and Edge Information

TL;DR

This paper tackles the problem of video deblurring under nonuniform, varying blur by introducing a sharp-frame prior in tandem with edge information. It presents SPEINet, an attention-based encoder-decoder framework that leverages sparse sharp frames via a lightweight logistic-regression detector and an edge-emphasizing module to guide reconstruction of neighboring blurred frames. The GoProRS dataset enables training and evaluation across a spectrum of sharp-frame ratios $r \in [0,0.5]$, improving generalization beyond fixed-ratio datasets. Empirical results across GoProO, GoProS, GoProRS, and BSD show that SPEINet delivers state-of-the-art PSNR/SSIM gains (average PSNR improvements around $+3.2\%$) with competitive inference times, demonstrating strong domain adaptability for real-world video deblurring tasks.

Abstract

Video deblurring is essential task for autonomous driving, facial recognition, and security surveillance. Traditional methods directly estimate motion blur kernels, often introducing artifacts and leading to poor results. Recent approaches utilize the detection of sharp frames within video sequences to enhance deblurring. However, existing datasets rely on fixed number of sharp frames, which may be too restrictive for some applications and may introduce a bias during model training. To address these limitations and enhance domain adaptability, this work first introduces GoPro Random Sharp (GoProRS), a new dataset where the the frequency of sharp frames within the sequence is customizable, allowing more diverse training and testing scenarios. Furthermore, it presents a novel video deblurring model, called SPEINet, that integrates sharp frame features into blurry frame reconstruction through an attention-based encoder-decoder architecture, a lightweight yet robust sharp frame detection and an edge extraction phase. Extensive experimental results demonstrate that SPEINet outperforms state-of-the-art methods across multiple datasets, achieving an average of +3.2% PSNR improvement over recent techniques. Given such promising results, we believe that both the proposed model and dataset pave the way for future advancements in video deblurring based on the detection of sharp frames.

Figures (8)

• Figure 1: Overview the SPEINet framework during the inference of the model. $F_{S}$ represents the sharp detection of the Stage 1. $F_{REE}$ represents the edge extraction of Stage 2; $F_{encB}$ and $F_{encS}$ represent the encoding performed at Stage 3. The modules in Search-Transfer represent the Stage 4. The $F_{dec}$ performs the reconstruction of Stage 5.
• Figure 2: Schema of Sharp Frames Detection. Logistic regression is trained to perform binary classification on each frame that can be sharp/blur. The $F_{S}$ searches for the closest sharp frame to $x_i$.
• Figure 3: Edge extraction obtained by applying the regularized-edge-emphasizing module $F_{\text{REE}}$ to the frame in \ref{['fig:subfig1']}.
• Figure 4: Three blocks encoder module. The encoder receives the blur frames $x_i$, the emphasized edges frames $c_i$, and any closest sharp frame $s_i$. Here $\varepsilon_{\theta} ( \cdot, k)$ denotes the encoder until block $k$. The triplets $\mathbf{u}_i$ and $\mathbf{v}_i$ are deduced by applying $\varepsilon(\cdot,3)$, frame-wise, to $\mathbf{x}_i$ and $\mathbf{c}_i$ respectively. While, if $s_i$ is found, $\mathbf{e}_i=(\varepsilon_\theta(s_i,1),\varepsilon_\theta(s_i,2),\varepsilon_\theta(s_i,3))$.
• Figure 5: Detail of Search-Transfer yang2020learning and Self-Search modules. Bis is based on the maximum similarity values and index coordinates identified in the search phase, the corresponding most similar features from the sharp features $e_i^{(3)}$ are selected for feature map reconstruction. The objective is to utilize the features of high-quality images to assist in the restoration or enhancement of details and textures in blurred images. Tra(2,3) denotes a transpose operation that swaps the second and third rows or columns. Flip(2) indicates a flipping operation along the second axis. INR refers to linear interpolation.