Motion-adaptive Separable Collaborative Filters for Blind Motion Deblurring

TL;DR

This work addresses real-world, spatially varying motion blur by shifting from residual reconstruction in feature space to an image-space filtering approach. It introduces the Motion-adaptive Separable Collaborative (MISC) Filter, consisting of a motion-guided alignment module and a separable collaborative filtering module, with a motion estimation network predicting a motion flow, mask, kernels, weights, and offsets to adaptively deblur regions. By exploring various coupling strategies between the motion estimation and residual reconstruction networks, the authors find that a shared encoder-decoder with filtering performed before reconstruction yields the best efficiency and performance. Empirical results on RealBlur and GoPro/HIDE demonstrate state-of-the-art performance on RealBlur with competitive runtime, and qualitative gains in texture and edge recovery, highlighting the practical potential of image-space filtering for blind motion deblurring. The work also provides ablation insights and releases code, supporting replication and extension in real-world deblurring tasks.

Abstract

Eliminating image blur produced by various kinds of motion has been a challenging problem. Dominant approaches rely heavily on model capacity to remove blurring by reconstructing residual from blurry observation in feature space. These practices not only prevent the capture of spatially variable motion in the real world but also ignore the tailored handling of various motions in image space. In this paper, we propose a novel real-world deblurring filtering model called the Motion-adaptive Separable Collaborative (MISC) Filter. In particular, we use a motion estimation network to capture motion information from neighborhoods, thereby adaptively estimating spatially-variant motion flow, mask, kernels, weights, and offsets to obtain the MISC Filter. The MISC Filter first aligns the motion-induced blurring patterns to the motion middle along the predicted flow direction, and then collaboratively filters the aligned image through the predicted kernels, weights, and offsets to generate the output. This design can handle more generalized and complex motion in a spatially differentiated manner. Furthermore, we analyze the relationships between the motion estimation network and the residual reconstruction network. Extensive experiments on four widely used benchmarks demonstrate that our method provides an effective solution for real-world motion blur removal and achieves state-of-the-art performance. Code is available at https://github.com/ChengxuLiu/MISCFilter

Figures (6)

• Figure 1: Illustration of other classical filters and our method. Red part is the center point of the filter in the image, and the green part is the reference point for generating it. Violet and orange parts are the weights and offsets in the filter, respectively, Blue part of our method indicates the motion-guided alignment along the motion direction (best viewed in color).
• Figure 2: Overview of proposed Motion-adaptive Separable Collaborative (MISC) Filter. MISC Filter focuses on removing various motions in image space. It inputs image feature $F$ obtained from a motion estimation network and generates filtered image $I"$ via a motion-guided alignment (MGA) module and a separable collaborative filtering (SCF) module.
• Figure 3: Design of different coupling strategies for motion estimation and residual reconstruction networks. In the ablation studies of Tab. \ref{['tab:f']}, we explore different strategies to generate filters and residuals.
• Figure 4: Visual results on Realblur-Jrim2020real, GoPronah2017deep, and HIDEshen2019human datasets. The method is shown at the bottom of each case. Zoom in to see better visualization.
• Figure 5: Visualization of ablation studies on MISC Filter.