HuPrior3R: Incorporating Human Priors for Better 3D Dynamic Reconstruction from Monocular Videos

TL;DR

This work tackles monocular dynamic 3D reconstruction with humans by fusing SMPL-based human priors and monocular depth through a cross-attention-based Feature Fusion Module in a hierarchical HuPrior3R architecture. A global processing stage builds scene-wide geometry while a refinement path crops and enhances small, detailed human regions via cross-module attention, guided by RANSAC-based depth alignment to ensure multi-modal consistency. The approach yields anatomically plausible human geometry, sharper boundaries, and improved camera pose stability, validated on diverse real and synthetic datasets (e.g., TUM Dynamics, GTA-IM, BEHAVE). Overall, HuPrior3R advances dynamic human reconstruction by integrating structured human priors with robust scene context, yielding superior depth, pose, and 3D point-cloud quality in challenging monocular videos.

Abstract

Monocular dynamic video reconstruction faces significant challenges in dynamic human scenes due to geometric inconsistencies and resolution degradation issues. Existing methods lack 3D human structural understanding, producing geometrically inconsistent results with distorted limb proportions and unnatural human-object fusion, while memory-constrained downsampling causes human boundary drift toward background geometry. To address these limitations, we propose to incorporate hybrid geometric priors that combine SMPL human body models with monocular depth estimation. Our approach leverages structured human priors to maintain surface consistency while capturing fine-grained geometric details in human regions. We introduce HuPrior3R, featuring a hierarchical pipeline with refinement components that processes full-resolution images for overall scene geometry, then applies strategic cropping and cross-attention fusion for human-specific detail enhancement. The method integrates SMPL priors through a Feature Fusion Module to ensure geometrically plausible reconstruction while preserving fine-grained human boundaries. Extensive experiments on TUM Dynamics and GTA-IM datasets demonstrate superior performance in dynamic human reconstruction.

Figures (7)

• Figure 1: The overview of our framework. The left shows the base HuPrior3R pipeline that processes image pairs through DUSt3R encoders, followed by a Feature Fusion Module that integrates image features, monocular depth features, and SMPL depth features via cross-attention mechanism and gated operations. The fused features are then processed through transformer decoders to produce initial point map reconstruction. The right illustrates HuPrior3R with Refinement, which is activated when humans occupy small pixel regions in the input images. The refinement stage crops human regions from the input images, upsamples the point clouds, and processes them through refinement decoders that incorporate cross-module attention between local crop features and preserved scene context. This hierarchical design enables high-fidelity human reconstruction while maintaining geometric consistency between human regions and the surrounding scene.
• Figure 2: Qualitative Comparison on Bonn and Tum dynamic Datasets. The red boxes highlight the comparison regions, demonstrating that our method outperforms existing works.
• Figure 3: Qualitative comparison of human reconstruction on HuPrior3R with and without refinement. Our refinement module produces more accurate human geometry and maintains better temporal consistency across frames in the cropped human regions.
• Figure 4: Detail of Feature Fusion Module.
• Figure 5: Qualitative ablation study on Prior Incorporation. The naive fusion approach (w/o FFM) introduces artifacts in background regions where SMPL priors are irrelevant, while our FFM preserves background quality.