Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SMURF: Continuous Dynamics for Motion-Deblurring Radiance Fields

Jungho Lee, Dogyoon Lee, Minhyeok Lee, Donghyung Kim, Sangyoun Lee

TL;DR

SMURF tackles the challenge of reconstructing sharp 3D scenes from motion-blurred images in neural radiance fields by modeling camera motion as a continuous process during exposure. It introduces a continuous motion blur kernel (CMBK) that evolves a latent representation with Neural-ODEs, then decodes this into a sequence of warped rays and weights integrated into a Tensorial Radiance Fields (TensoRF) backbone with explicit voxel grids. Key innovations include a chrono-view embedding to fuse time and view, a residual momentum regularization, and an output suppression loss to stabilize optimization. Empirical results on synthetic and real motion-blur datasets show state-of-the-art novel-view synthesis performance (higher PSNR, SSIM and lower LPIPS) compared with Deblur-NeRF and related methods, enabling robust 3D reconstruction under realistic motion blur. The approach promises practical impact for photo-realistic view synthesis in photography, AR/VR, and robotics where motion blur is inevitable.

Abstract

Neural radiance fields (NeRF) has attracted considerable attention for their exceptional ability in synthesizing novel views with high fidelity. However, the presence of motion blur, resulting from slight camera movements during extended shutter exposures, poses a significant challenge, potentially compromising the quality of the reconstructed 3D scenes. To effectively handle this issue, we propose sequential motion understanding radiance fields (SMURF), a novel approach that models continuous camera motion and leverages the explicit volumetric representation method for robustness to motion-blurred input images. The core idea of the SMURF is continuous motion blurring kernel (CMBK), a module designed to model a continuous camera movements for processing blurry inputs. Our model is evaluated against benchmark datasets and demonstrates state-of-the-art performance both quantitatively and qualitatively.

SMURF: Continuous Dynamics for Motion-Deblurring Radiance Fields

TL;DR

SMURF tackles the challenge of reconstructing sharp 3D scenes from motion-blurred images in neural radiance fields by modeling camera motion as a continuous process during exposure. It introduces a continuous motion blur kernel (CMBK) that evolves a latent representation with Neural-ODEs, then decodes this into a sequence of warped rays and weights integrated into a Tensorial Radiance Fields (TensoRF) backbone with explicit voxel grids. Key innovations include a chrono-view embedding to fuse time and view, a residual momentum regularization, and an output suppression loss to stabilize optimization. Empirical results on synthetic and real motion-blur datasets show state-of-the-art novel-view synthesis performance (higher PSNR, SSIM and lower LPIPS) compared with Deblur-NeRF and related methods, enabling robust 3D reconstruction under realistic motion blur. The approach promises practical impact for photo-realistic view synthesis in photography, AR/VR, and robotics where motion blur is inevitable.

Abstract

Neural radiance fields (NeRF) has attracted considerable attention for their exceptional ability in synthesizing novel views with high fidelity. However, the presence of motion blur, resulting from slight camera movements during extended shutter exposures, poses a significant challenge, potentially compromising the quality of the reconstructed 3D scenes. To effectively handle this issue, we propose sequential motion understanding radiance fields (SMURF), a novel approach that models continuous camera motion and leverages the explicit volumetric representation method for robustness to motion-blurred input images. The core idea of the SMURF is continuous motion blurring kernel (CMBK), a module designed to model a continuous camera movements for processing blurry inputs. Our model is evaluated against benchmark datasets and demonstrates state-of-the-art performance both quantitatively and qualitatively.
Paper Structure (18 sections, 12 equations, 3 figures, 2 tables)

This paper contains 18 sections, 12 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Blind Deblurring for 3D Scene.
  4. Tensorial Radiance Fields (TensoRF).
  5. Method
  6. Continuous Motion Blur Kernel
  7. Continuous Latent Space Modeling for Camera Motion.
  8. Motion-Blurring Kernel Generation.
  9. Residual Momentum.
  10. Output Suppression Loss.
  11. Experiments
  12. Implementation Details
  13. Datasets.
  14. Novel View Synthesis Results
  15. Ablation Study
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Blurring process and our sequential kernel generation. A blurry image $I_{blur}$ is acquired as the camera moves over the exposure time, with images $I_{t}$ captured at each camera pose being composited together. Our approach estimates warped rays along the continuous camera motion trajectory.
  • Figure 2: Method overview of the SMURF. The latent feature of initial ray is extended into an IVP with a parameterized derivative function $f_{\phi}$. Then, it is solved by Neural-ODEs along with given time $t$, obtaining latent features for all warped rays.
  • Figure 3: Qualitative comparisons on synthetic scenes and real-world scenes.