Deep Hybrid Camera Deblurring for Smartphone Cameras

TL;DR

Deep Hybrid Camera Deblurring (HCDeblur) tackles smartphone blur by exploiting a hybrid camera setup that records a long-exposure wide image $W$ and a burst of short-exposure ultra-wide images $\mathbf{U}$ to recover a sharp output. It introduces HC-DNet, which builds blur kernels from burst motion to deblur toward $W_D$, and HC-FNet, which refines $W_D$ using the burst as references to yield $W_F$. The HCBlur dataset, with HCBlur-Syn and HCBlur-Real, enables training and robust evaluation across synthetic and real data. Experiments demonstrate state-of-the-art deblurring performance and clear advantages of burst-informed kernel estimation and refinement for practical smartphone deployment.

Abstract

Mobile cameras, despite their significant advancements, still have difficulty in low-light imaging due to compact sensors and lenses, leading to longer exposures and motion blur. Traditional blind deconvolution methods and learning-based deblurring methods can be potential solutions to remove blur. However, achieving practical performance still remains a challenge. To address this, we propose a learning-based deblurring framework for smartphones, utilizing wide and ultra-wide cameras as a hybrid camera system. We simultaneously capture a long-exposure wide image and short-exposure burst ultra-wide images, and utilize the burst images to deblur the wide image. To fully exploit burst ultra-wide images, we present HCDeblur, a practical deblurring framework that includes novel deblurring networks, HC-DNet and HC-FNet. HC-DNet utilizes motion information extracted from burst images to deblur a wide image, and HC-FNet leverages burst images as reference images to further enhance a deblurred output. For training and evaluating the proposed method, we introduce the HCBlur dataset, which consists of synthetic and real-world datasets. Our experiments demonstrate that HCDeblur achieves state-of-the-art deblurring quality. Code and datasets are available at https://cg.postech.ac.kr/research/HCDeblur.

Figures (6)

• Figure 1: Overview of HCDeblur. Our framework takes a long-exposure wide image $W$ and a burst of short-exposure ultra-wide images $\mathbf{U}$ as inputs. We estimate a homography matrix $\hat{H}$ for aligning $\mathbf{U}$ in the FOV alignment (Sec. \ref{['sec:foval']}) and compute pixel-wise motion trajectories $\mathbf{P}$ (Sec. \ref{['sec:PT']}). HC-DNet performs kernel-based deblurring by exploiting blur kernels $\mathbf{K}$ constructed from $\mathbf{P}$ (Sec. \ref{['sec:hc-dnet']}). After deblurring, an additional alignment step is adopted to align the burst images to the deblurred wide image $W_{D}$. HC-FNet further enhances the deblurred image by using the entire sequence of the burst images as reference images (Sec. \ref{['sec:hc-fnet']}).
• Figure 2: Blur kernels constructed from pixel-wise motion trajectories.
• Figure 3: Architecture of HC-DNet.
• Figure 4: Architecture of HC-FNet.
• Figure 5: Qualitative comparisons on the HCBlur-Syn dataset. We compare HCDeblur with NAFNet-64 chen2022simple, NAFNet-Ref, and MotionETR zhang2021exposure.