Blur2seq: Blind Deblurring and Camera Trajectory Estimation from a Single Camera Motion-blurred Image

TL;DR

This work tackles motion blur from camera shake by jointly estimating the latent sharp image and the underlying camera trajectory from a single blurry image. It grounds the approach in the Projective Motion Blur Model (PMBM) and Exposure Trajectory Recovery (ETR), representing blur via per-pixel offsets and a differentiable blur creation module, and predicts a full 3D rotation trajectory with a two-branch Trajectory Prediction Network. A model-based Restoration Network unrolls ADMM iterations to recover the latent image from the blurry input and the PMBM-derived degradation, with a reblur loss used to refine the trajectory post-inference; this yields a transparent, interpretable blur model and enables video generation of the acquisition process. Experiments on synthetic and real datasets show state-of-the-art or competitive performance, especially for severe or spatially variant blur, and the method offers practical benefits by providing the motion path and a recoverable sequence of sharp frames along with efficient, differentiable components and a strong integration of learned priors with physics-based degradation models.

Abstract

Motion blur caused by camera shake, particularly under large or rotational movements, remains a major challenge in image restoration. We propose a deep learning framework that jointly estimates the latent sharp image and the underlying camera motion trajectory from a single blurry image. Our method leverages the Projective Motion Blur Model (PMBM), implemented efficiently using a differentiable blur creation module compatible with modern networks. A neural network predicts a full 3D rotation trajectory, which guides a model-based restoration network trained end-to-end. This modular architecture provides interpretability by revealing the camera motion that produced the blur. Moreover, this trajectory enables the reconstruction of the sequence of sharp images that generated the observed blurry image. To further refine results, we optimize the trajectory post-inference via a reblur loss, improving consistency between the blurry input and the restored output. Extensive experiments show that our method achieves state-of-the-art performance on both synthetic and real datasets, particularly in cases with severe or spatially variant blur, where end-to-end deblurring networks struggle. Code and trained models are available at https://github.com/GuillermoCarbajal/Blur2Seq/

Figures (8)

• Figure 1: The Trajectory Prediction Network estimates the camera motion during image acquisition. According to the Projective Motion Blur Model, this trajectory defines the image degradation, which we characterize as a set of pixel-specific time-dependent offsets. A model-based restoration network then takes the predicted offsets together with the blurry image to produce the restored image. Finally, the Video Generation Module uses the restoration and the pixel displacements to synthesize a video of the acquisition process.
• Figure 2: Motion Trajectory Prediction Network. The U-Net type network has two branches. One of the branches predicts the pitch and the yaw, and the other estimates the roll of the camera.
• Figure 3: Blur Creation Module proposed by zhang2021exposure. The module receives the offsets $\Delta(\mathbf{i})=(\Delta^{(0)},\Delta^{(1)}, \ldots, \Delta^{(T-1)})$ for each pixel $\mathbf{i}$. With the offsets associated to time step $t_n$ ($\Delta^{(t_n)}$), the module generates an image $\mathbf{u}^{(t_n)}$ using deformable convolutions ($*^d$). The blurry image is the average of the $T$ images $\mathbf{u}^{(t_n)}$.
• Figure 4: Efficient Blur Creation Module. The module receives the offsets $\Delta(\mathbf{i})=(\Delta^{0},\Delta^{1}, \ldots, \Delta^{T-1})$ for each pixel $\mathbf{i}$. The offsets are first rearranged into a $\sqrt{T}$ square matrix, and then the original offsets values $\Delta(\mathbf{i})$ modified to compensate the displacement from the center in the matrix ($\tilde{\Delta}(\mathbf{i})=\Delta(\mathbf{i})+\mathbf{p}$). The blurry image is then generated as a deformable convolution ($\mathbf{*}^d$) with a size $\sqrt{T}$ filter with constant values $(1/T)$ and sampling locations determined by the the modified offsets $\tilde{\Delta}(\mathbf{i})=(\tilde{\Delta}^{0},\tilde{\Delta}^{1}, \ldots, \tilde{\Delta}^{T-1})$.
• Figure 5: Evolution of reblur and restoration PSNRs during the trajectory optimization step for a sample image.