ExBluRF: Efficient Radiance Fields for Extreme Motion Blurred Images

TL;DR

ExBluRF addresses the challenge of recovering sharp 3D scenes from extreme motion blurred multi-view images, where standard NeRF-based optimization struggles due to blur-induced ambiguity. It introduces a 6-DOF camera trajectory blur model and couples it with voxel-based radiance fields, enabling joint optimization of sharp geometry and camera motion without heavy MLP-based networks. The method uses Bézier-curve trajectories to represent sub-frame poses and optimizes both the latent radiance field and the per-view trajectories, with an ExBlur dataset providing real blurred and ground-truth sharp sequences for evaluation. Empirical results show sharper reconstructions and superior novel-view synthesis at much lower training time and memory usage than prior approaches, demonstrating practical scalability for low-light or hand-held capture scenarios.

Abstract

We present ExBluRF, a novel view synthesis method for extreme motion blurred images based on efficient radiance fields optimization. Our approach consists of two main components: 6-DOF camera trajectory-based motion blur formulation and voxel-based radiance fields. From extremely blurred images, we optimize the sharp radiance fields by jointly estimating the camera trajectories that generate the blurry images. In training, multiple rays along the camera trajectory are accumulated to reconstruct single blurry color, which is equivalent to the physical motion blur operation. We minimize the photo-consistency loss on blurred image space and obtain the sharp radiance fields with camera trajectories that explain the blur of all images. The joint optimization on the blurred image space demands painfully increasing computation and resources proportional to the blur size. Our method solves this problem by replacing the MLP-based framework to low-dimensional 6-DOF camera poses and voxel-based radiance fields. Compared with the existing works, our approach restores much sharper 3D scenes from challenging motion blurred views with the order of 10 times less training time and GPU memory consumption.

Figures (6)

• Figure 1: Given a set of extremely blurred multi-view images (a), our method restores sharp radiance fields and renders clearly deblurred novel views (b).
• Figure 2: Training time and GPU memory consumption on "Camellia" shown in Fig \ref{['fig:teaser']}. Our method, ExBluRF, significantly improves efficiency on both the training time and memory cost with better deblurring performance. $N$ is the number of samples (kernel size) to reconstruct blurry color.
• Figure 3: The overview of ExBluRF. We incorporate the physical operation that generates camera motion blur in the volume rendering of radiance fields. The blurry RGB color is reproduced by accumulating the rays along the estimated camera trajectory. By minimizing the photo-consistency loss between the accumulated color and input blurry RGB, we obtain sharp radiance fields and the camera trajectories that explain the motion blur of training views. We adopt voxel-based radiance fields to deal with explosive computation when optimizing extremely motion blurred scenes.
• Figure 4: (a) Blurry training view of synthetic data. (b) Visualization of the GT (green) and estimated (red) trajectory of (a) by increasing the training iterations. Our blur model converges well to the GT trajectory without sophisticated initialization.
• Figure 5: Qualitative comparison of deblurring on ExBlur dataset.