MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting

TL;DR

MotionGS tackles dynamic scene reconstruction by introducing explicit motion priors to deformable 3D Gaussian Splatting. It decouples 2D optical flow into camera flow and motion flow, using the motion component to supervise Gaussian deformation through a differentiable Gaussian flow, and couples this with a camera pose refinement loop that alternates updates to Gaussians and poses. The method achieves state-of-the-art results on NeRF-DS and HyperNeRF, showing improved rendering quality, motion handling, and robustness in challenging monocular dynamic scenes. It is compatible with existing 3DGS architectures and points toward future pose-free deformable 3DGS, enabling robust high-quality dynamic reconstructions.

Abstract

Dynamic scene reconstruction is a long-term challenge in the field of 3D vision. Recently, the emergence of 3D Gaussian Splatting has provided new insights into this problem. Although subsequent efforts rapidly extend static 3D Gaussian to dynamic scenes, they often lack explicit constraints on object motion, leading to optimization difficulties and performance degradation. To address the above issues, we propose a novel deformable 3D Gaussian splatting framework called MotionGS, which explores explicit motion priors to guide the deformation of 3D Gaussians. Specifically, we first introduce an optical flow decoupling module that decouples optical flow into camera flow and motion flow, corresponding to camera movement and object motion respectively. Then the motion flow can effectively constrain the deformation of 3D Gaussians, thus simulating the motion of dynamic objects. Additionally, a camera pose refinement module is proposed to alternately optimize 3D Gaussians and camera poses, mitigating the impact of inaccurate camera poses. Extensive experiments in the monocular dynamic scenes validate that MotionGS surpasses state-of-the-art methods and exhibits significant superiority in both qualitative and quantitative results. Project page: https://ruijiezhu94.github.io/MotionGS_page

Figures (12)

• Figure 1: (a) Gaussian flow under different supervision. We model Gaussian flow under the supervision of optical flow and motion flow respectively. The latter can produce a more direct description of object motion, thereby effectively guiding the deformation of 3D Gaussians. (b) The decoupling of optical flow. We decouple the optical flow into motion flow which is only related to object motion and camera flow which is only related to camera motion.
• Figure 2: The overall architecture of MotionGS. It can be viewed as two data streams: (1) The 2D data stream utilizes the optical flow decoupling module to obtain the motion flow as the 2D motion prior; (2) The 3D data stream involves the deformation and transformation of Gaussians to render the image for the next frame. During training, we alternately optimize 3DGS and camera poses through the camera pose refinement module.
• Figure 3: Flow calculation.
• Figure 4: Pose refinement on iterative training.
• Figure 5: Qualitative comparison on NeRF-DS dataset. Refer to \ref{['fig: nerfds_full']} for more scenes.