PEGS: Physics-Event Enhanced Large Spatiotemporal Motion Reconstruction via 3D Gaussian Splatting

TL;DR

PEGS addresses the challenge of reconstructing large, rigid motion over long spatiotemporal horizons from blurry monocular input by fusing physical priors with high-temporal-resolution event streams within a 3D Gaussian Splatting representation. The method introduces a triple-level supervision scheme that enforces acceleration consistency, aligns with event-driven cues, and regularizes through Kalman fusion, complemented by a motion-aware simulated annealing schedule to improve convergence. A first RGB-Event paired dataset for natural fast rigid motion across diverse scenarios is provided, and PEGS demonstrates state-of-the-art performance in both deblurring and SE(3) motion recovery on synthetic and real data. The work offers a principled, physics-grounded pathway for robust dynamic scene understanding and lays groundwork for accurate 4D motion synthesis and analysis in challenging real-world settings.

Abstract

Reconstruction of rigid motion over large spatiotemporal scales remains a challenging task due to limitations in modeling paradigms, severe motion blur, and insufficient physical consistency. In this work, we propose PEGS, a framework that integrates Physical priors with Event stream enhancement within a 3D Gaussian Splatting pipeline to perform deblurred target-focused modeling and motion recovery. We introduce a cohesive triple-level supervision scheme that enforces physical plausibility via an acceleration constraint, leverages event streams for high-temporal resolution guidance, and employs a Kalman regularizer to fuse multi-source observations. Furthermore, we design a motion-aware simulated annealing strategy that adaptively schedules the training process based on real-time kinematic states. We also contribute the first RGB-Event paired dataset targeting natural, fast rigid motion across diverse scenarios. Experiments show PEGS's superior performance in reconstructing motion over large spatiotemporal scales compared to mainstream dynamic methods.

Figures (7)

• Figure 1: PEGS takes a monocular blurry video and event streams as input to perform 4D motion recovery. The framework first focuses on the target for deblurred 3D Gaussian reconstruction, then estimates the SE-3 transformations of the motion sequence. By integrating physical priors with event enhancement, PEGS effectively reconstructs large motions, producing outputs applicable to downstream tasks.
• Figure 2: PEGS reconstructs a target-focused 3D Gaussian scene from blurry images and then estimates its full SE(3) motion trajectory. This is achieved by combining event-based deblurring with a triple-level motion supervision strategy that enforces acceleration consistency, event stream alignment, and Kalman regularization. A motion-aware simulated annealing scheduler further boosts training convergence.
• Figure 3: MSA strategy. Bottom: An object accelerates in a constant force field, with the velocity $\bm{v}$ and displacement $\bm{s}$ vary at each timestamp. Top: The red curve traces the initial learning rate, and blue points signify its value after exponential decay.
• Figure 4: Schematic of RGB-Event spatiotemporal synchronization acquisition device (left) and imaging process (right).
• Figure 5: Qualitative comparison on the synthetic dataset. For clearer results, please refer to the video in the supplementary material.