PAGE-4D: Disentangled Pose and Geometry Estimation for VGGT-4D Perception

TL;DR

PAGE-4D proposes a dynamics-aware extension of VGGT to dynamic scenes by introducing a learned dynamics mask and a three-stage cross-attention aggregator that suppresses dynamic regions for camera pose estimation while leveraging motion cues for depth and 3D reconstruction. By fine-tuning only the middle transformer layers and employing a lightweight, self-supervised mask head, PAGE-4D achieves state-of-the-art results across video depth, monocular depth, camera pose, and point-map reconstruction in dynamic benchmarks. The approach effectively reconciles the static epipolar constraints needed for pose with the dynamic information useful for geometry, delivering robust 4D perception with minimal runtime overhead. These contributions demonstrate a scalable path to reliable 4D understanding in real-world, motion-rich environments.

Abstract

Recent 3D feed-forward models, such as the Visual Geometry Grounded Transformer (VGGT), have shown strong capability in inferring 3D attributes of static scenes. However, since they are typically trained on static datasets, these models often struggle in real-world scenarios involving complex dynamic elements, such as moving humans or deformable objects like umbrellas. To address this limitation, we introduce PAGE-4D, a feedforward model that extends VGGT to dynamic scenes, enabling camera pose estimation, depth prediction, and point cloud reconstruction -- all without post-processing. A central challenge in multi-task 4D reconstruction is the inherent conflict between tasks: accurate camera pose estimation requires suppressing dynamic regions, while geometry reconstruction requires modeling them. To resolve this tension, we propose a dynamics-aware aggregator that disentangles static and dynamic information by predicting a dynamics-aware mask -- suppressing motion cues for pose estimation while amplifying them for geometry reconstruction. Extensive experiments show that PAGE-4D consistently outperforms the original VGGT in dynamic scenarios, achieving superior results in camera pose estimation, monocular and video depth estimation, and dense point map reconstruction.

Figures (5)

• Figure 1: PAGE-4D takes a sequence of RGB images depicting a dynamic scene as input and simultaneously predicts the corresponding camera parameters and 3D geometry information—all within a fraction of a second. Compared to VGGT, PAGE-4D produces denser and more accurate point cloud reconstructions with better depth estimation quality. (Best viewed in PDF.)
• Figure 2: Motivating illustration: (a) In static scenes, geometric consistency is preserved across frames, while in dynamic scenes, moving objects violate this consistency. (b) Visualization of VGGT attention maps from the 5st, 12nd, 18th, and 24th layers of global attention block with the method in caron2021emerging. Attention values are visualized using a white-to-red color map, with white indicating low values and red indicating high values. VGGT tends to ignore dynamic content during the feed-forward process, which motivates our design of the dynamics-aware mask.
• Figure 3: Fine-tuning strategy: Instead of fine-tuning the entire VGGT architecture, we adapt only the middle 10 layers of the global attention mechanism, which are most critical for cross-frame information fusion. To further address dynamic scenes, we introduce a dynamics-aware aggregator that predicts a mask to disentangle dynamic and static content.
• Figure 4: Qualitative Comparison of Point Cloud Estimation on the Bonn & Sintel: As shown in the figure, our method effectively captures the geometric structure in scenarios with complex motion, whereas VGGT produces fragmented and inconsistent geometry. (Best viewed in PDF.)
• Figure 5: Qualitative Results of Point Cloud Estimation.PAGE-4D can estimate camera poses and depth maps from RGB inputs, even in the presence of dynamic objects. (Best viewed in PDF.)