PMGS: Reconstruction of Projectile Motion Across Large Spatiotemporal Spans via 3D Gaussian Splatting

TL;DR

PMGS addresses the challenge of reconstructing complex rigid projectile motion over large spatiotemporal spans from monocular video by introducing a physics-informed two-stage pipeline. It first performs Target Modeling with 3D Gaussian Splatting to create a stable, object-centered representation, then conducts Motion Recovery by estimating per-frame $SE(3)$ poses under an acceleration consistency constraint, aided by Dynamic Simulated Annealing and a cross-modal Kalman fusion scheme. The key contributions include a direct acceleration prior linking Newtonian dynamics to pose estimation, a dynamic training schedule that adapts to velocity-displacement changes, and a Kalman-based fusion framework that robustly combines optical-flow and learned observations. Across synthetic and real datasets, PMGS outperforms state-of-the-art dynamic rendering methods in both video reconstruction and full-motion recovery, enabling more physically plausible long-span motion understanding from monocular cues.

Abstract

Modeling complex rigid motion across large spatiotemporal spans remains an unresolved challenge in dynamic reconstruction. Existing paradigms are mainly confined to short-term, small-scale deformation and offer limited consideration for physical consistency. This study proposes PMGS, focusing on reconstructing Projectile Motion via 3D Gaussian Splatting. The workflow comprises two stages: 1) Target Modeling: achieving object-centralized reconstruction through dynamic scene decomposition and an improved point density control; 2) Motion Recovery: restoring full motion sequences by learning per-frame SE(3) poses. We introduce an acceleration consistency constraint to bridge Newtonian mechanics and pose estimation, and design a dynamic simulated annealing strategy that adaptively schedules learning rates based on motion states. Futhermore, we devise a Kalman fusion scheme to optimize error accumulation from multi-source observations to mitigate disturbances. Experiments show PMGS's superior performance in reconstructing high-speed nonlinear rigid motion compared to mainstream dynamic methods.

Figures (5)

• Figure 1: Left: Current paradigms focus on small-scale deformation reconstruction. Right: PMGS explores complex rigid motion modeling across large spatiotemporal spans.
• Figure 2: Overview of PMGS. We first segment the target, then decompose the motion through centralization to transform the dynamic scene into a static one. For modeling, we learn a set of Gaussian kernels and align them at the original scale with a set of learnable affine transformations. In motion recovery, we estimate the target's SE(3) transformation frame by frame, and comprehensively improve tracking accuracy by integrating physics-enhanced strategies.
• Figure 3: DSA strategy. Right: An object accelerates in a constant gravity field, with different velocities $v$ and displacements $s$ corresponding to each timestamp. Left: Red curve represents the initial learning rate. Blue points show the final learning rate after exponential decay.
• Figure 4: Qualitative comparison on both synthetic and real datasets. PMGS generalizes well across different scenarios and accurately reconstructs full-sequence motion. The displayed results are uniformly sampled, for complete and coherent motion recovery, please refer to the video on the project page.
• Figure 5: Ablation results for different comparison models. PMGS in its complete form exhibits superior stability.