Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CRiM-GS: Continuous Rigid Motion-Aware Gaussian Splatting from Motion-Blurred Images

Jungho Lee, Donghyeong Kim, Dogyoon Lee, Suhwan Cho, Minhyeok Lee, Sangyoun Lee

TL;DR

Motivated by motion blur during exposure, this work targets 3D scene reconstruction from motion-blurred imagery. It introduces CRiM-GS, a continuous rigid motion-aware Gaussian splatting framework that models continuous camera trajectories via neural ODEs, enforces rigid-body regularization, and corrects nonlinear distortions with an adaptive distortion-aware transformation, all within a differentiable Gaussian splatting pipeline. The method achieves state-of-the-art results on synthetic and real blurred datasets, validated by objective metrics and qualitative visualizations, while maintaining real-time rendering speed through Mip-Splatt ing. This approach broadens the applicability of neural rendering to realistic capture conditions and offers robust deblurring-driven 3D reconstruction for AR/VR pipelines.

Abstract

3D Gaussian Splatting (3DGS) has gained significant attention for their high-quality novel view rendering, motivating research to address real-world challenges. A critical issue is the camera motion blur caused by movement during exposure, which hinders accurate 3D scene reconstruction. In this study, we propose CRiM-GS, a \textbf{C}ontinuous \textbf{Ri}gid \textbf{M}otion-aware \textbf{G}aussian \textbf{S}platting that reconstructs precise 3D scenes from motion-blurred images while maintaining real-time rendering speed. Considering the complex motion patterns inherent in real-world camera movements, we predict continuous camera trajectories using neural ordinary differential equations (ODE). To ensure accurate modeling, we employ rigid body transformations with proper regularization, preserving object shape and size. Additionally, we introduce an adaptive distortion-aware transformation to compensate for potential nonlinear distortions, such as rolling shutter effects, and unpredictable camera movements. By revisiting fundamental camera theory and leveraging advanced neural training techniques, we achieve precise modeling of continuous camera trajectories. Extensive experiments demonstrate state-of-the-art performance both quantitatively and qualitatively on benchmark datasets.

CRiM-GS: Continuous Rigid Motion-Aware Gaussian Splatting from Motion-Blurred Images

TL;DR

Motivated by motion blur during exposure, this work targets 3D scene reconstruction from motion-blurred imagery. It introduces CRiM-GS, a continuous rigid motion-aware Gaussian splatting framework that models continuous camera trajectories via neural ODEs, enforces rigid-body regularization, and corrects nonlinear distortions with an adaptive distortion-aware transformation, all within a differentiable Gaussian splatting pipeline. The method achieves state-of-the-art results on synthetic and real blurred datasets, validated by objective metrics and qualitative visualizations, while maintaining real-time rendering speed through Mip-Splatt ing. This approach broadens the applicability of neural rendering to realistic capture conditions and offers robust deblurring-driven 3D reconstruction for AR/VR pipelines.

Abstract

3D Gaussian Splatting (3DGS) has gained significant attention for their high-quality novel view rendering, motivating research to address real-world challenges. A critical issue is the camera motion blur caused by movement during exposure, which hinders accurate 3D scene reconstruction. In this study, we propose CRiM-GS, a \textbf{C}ontinuous \textbf{Ri}gid \textbf{M}otion-aware \textbf{G}aussian \textbf{S}platting that reconstructs precise 3D scenes from motion-blurred images while maintaining real-time rendering speed. Considering the complex motion patterns inherent in real-world camera movements, we predict continuous camera trajectories using neural ordinary differential equations (ODE). To ensure accurate modeling, we employ rigid body transformations with proper regularization, preserving object shape and size. Additionally, we introduce an adaptive distortion-aware transformation to compensate for potential nonlinear distortions, such as rolling shutter effects, and unpredictable camera movements. By revisiting fundamental camera theory and leveraging advanced neural training techniques, we achieve precise modeling of continuous camera trajectories. Extensive experiments demonstrate state-of-the-art performance both quantitatively and qualitatively on benchmark datasets.
Paper Structure (36 sections, 21 equations, 8 figures, 7 tables)

This paper contains 36 sections, 21 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Rendering
  4. Neural Rendering from Blurry Images
  5. Preliminary
  6. 3D Scene Blind Deblurring
  7. 3D Gaussian Splatting
  8. Neural Ordinary Differential Equations
  9. Method
  10. CRiM-GS Framework
  11. Continuous Rigid Body Motion
  12. Adaptive Distortion-Aware Transformation
  13. Optimization
  14. Pixel-wise Weight.
  15. Objective.
  16. ...and 21 more sections

Figures (8)

  • Figure 1: Pipeline of CRiM-GS. The input camera contains the image index and the initial camera pose information. CRiM-GS iteratively solves the ODE using the encoded image index and the neural derivative presented in \ref{['sec:rigid']} and \ref{['sec:deformable']}, and obtains $N$ transformed camera poses after decoding. Then, $N$ images are rendered through 3DGS, and a blurry image is obtained through the pixel-wise weighted-sum presented in \ref{['sec:optimize']}
  • Figure 2: Qualitative comparison on the synthetic and real-world scenes.
  • Figure 3: Camera Trajectory Visualization. Red cones stand for camera poses, and the images below are the output blurry images.
  • Figure 4: Rendering Results on Rolling Shutter Effect Dataset seiskari2024gaussian
  • Figure 5: Camera motion trajectory predicted by CRiM-GS for input images with significant blur.
  • ...and 3 more figures