Recovering 3D Shapes from Ultra-Fast Motion-Blurred Images

TL;DR

This work tackles recovering 3D shapes from ultra-fast motion-blurred images, where traditional 3D reconstruction fails due to severe blur. It introduces a differentiable, rasterization-based inverse rendering pipeline that features a fast barycentric coordinate solver, enabling efficient aggregation of motion blur across many frames. The method supports translational and rotational motion, providing both mesh-based (translation) and SDF-based (rotation) shape optimization from multi-view blurred imagery, with extensive synthetic and real-world validation and ablations showing clear speed and accuracy gains over SoftRas and Nvdiffrast. Together, these contributions advance high-speed 3D reconstruction, offering practical capabilities for fast-moving objects in sports, machinery, and robotics with improved efficiency and gradient quality for optimization.

Abstract

We consider the problem of 3D shape recovery from ultra-fast motion-blurred images. While 3D reconstruction from static images has been extensively studied, recovering geometry from extreme motion-blurred images remains challenging. Such scenarios frequently occur in both natural and industrial settings, such as fast-moving objects in sports (e.g., balls) or rotating machinery, where rapid motion distorts object appearance and makes traditional 3D reconstruction techniques like Multi-View Stereo (MVS) ineffective. In this paper, we propose a novel inverse rendering approach for shape recovery from ultra-fast motion-blurred images. While conventional rendering techniques typically synthesize blur by averaging across multiple frames, we identify a major computational bottleneck in the repeated computation of barycentric weights. To address this, we propose a fast barycentric coordinate solver, which significantly reduces computational overhead and achieves a speedup of up to 4.57x, enabling efficient and photorealistic simulation of high-speed motion. Crucially, our method is fully differentiable, allowing gradients to propagate from rendered images to the underlying 3D shape, thereby facilitating shape recovery through inverse rendering. We validate our approach on two representative motion types: rapid translation and rotation. Experimental results demonstrate that our method enables efficient and realistic modeling of ultra-fast moving objects in the forward simulation. Moreover, it successfully recovers 3D shapes from 2D imagery of objects undergoing extreme translational and rotational motion, advancing the boundaries of vision-based 3D reconstruction. Project page: https://maxmilite.github.io/rec-from-ultrafast-blur/

Figures (15)

• Figure 1: Ultra-fast motion blur is common in real-world scenarios. Top: A ball undergoing translational motion rozumnyi2021defmo. Bottom: A spinning top in rotation bacher2014spin. In this paper, our goal is to recover 3D shapes from the high-speed translational and rotational motion.
• Figure 2: Motion blur is typically synthesized by rendering and averaging multiple frames. However, for extreme motion, a large number of frames are required to achieve realistic results. Here, we illustrate a bicycle undergoing extreme translation. Noticeable artifacts appear when using fewer samples, and at least 50 frames are needed to produce a realistic motion-blurred image.
• Figure 3: Forward rendering & backward gradient visualization for ultra-fast motion-blur synthesis. Our rendered images and gradients exhibit a high degree of similarity to those generated by SoftRas across various motion cases and sample numbers. Scene settings and render details are provided in \ref{['sec:supp:visualization-details', 'sec:supp:scene-settings']}.
• Figure 4: Forward + gradient computation timing results. The slope represents the average time of sampling once. The lower the better. Our method achieves speedups of up to 4.57$\times$ and 1.23$\times$ compared to SoftRas and Nvdiffrast, respectively.
• Figure 5: Qualitative results on geometry and color optimization. (a) One of blurred input images. (b) State of the Art rozumnyi2021shape result. (c) Our optimization result. (d) Ground-truth object. Our method yields significant superior results than the state-of-the-art work. Note that the object trajectories are synthesized to test the solver's robustness to diverse motion vectors, rather than simulating realistic physical dynamics.