Gyro-based Neural Single Image Deblurring

TL;DR

This work tackles the ill-posed problem of single-image deblurring by leveraging gyro sensor data to inform motion, addressing real-world gyro errors with a dedicated network design. GyroDeblurNet uses a novel camera motion field embedding and two blocks—gyro refinement and gyro deblurring—to robustly extract sharp details even when gyro data are noisy. A curriculum-learning strategy gradually introduces gyro errors during training, and two realistic datasets (GyroBlur-Synth and GyroBlur-Real) provide synthetic and real-world benchmarks. The approach achieves state-of-the-art deblurring quality while maintaining efficiency, demonstrating strong robustness to sensor errors and cross-device applicability with practical impact for mobile photography and vision tasks.

Abstract

In this paper, we present GyroDeblurNet, a novel single-image deblurring method that utilizes a gyro sensor to resolve the ill-posedness of image deblurring. The gyro sensor provides valuable information about camera motion that can improve deblurring quality. However, exploiting real-world gyro data is challenging due to errors from various sources. To handle these errors, GyroDeblurNet is equipped with two novel neural network blocks: a gyro refinement block and a gyro deblurring block. The gyro refinement block refines the erroneous gyro data using the blur information from the input image. The gyro deblurring block removes blur from the input image using the refined gyro data and further compensates for gyro error by leveraging the blur information from the input image. For training a neural network with erroneous gyro data, we propose a training strategy based on the curriculum learning. We also introduce a novel gyro data embedding scheme to represent real-world intricate camera shakes. Finally, we present both synthetic and real-world datasets for training and evaluating gyro-based single image deblurring. Our experiments demonstrate that our approach achieves state-of-the-art deblurring quality by effectively utilizing erroneous gyro data.

Figures (10)

• Figure 1: Our method shows robust performance even with errors in real-world gyro data. (a) A real-world blurry image. (b) Gyro data visualization onto the blurry image. The gyro data do not perfectly match the blur trajectories due to errors. (c) Our deblurred result.
• Figure 2: Camera motion field construction. (a) Computing homography and warped pixel coordinates. (b) Constructing camera motion field by stacking motion vectors.
• Figure 3: Network architecture of GyroDeblurNet.
• Figure 4: Detailed architectures of the modules in the GyroDeblurNet. (a) Architecture of the gyro refinement block. (b) Architecture of the gyro deblurring block.
• Figure 5: Qualitative comparison on GyroBlur-Synth. The red curves overlaid on the blurred images visualize the input gyro data.