PRGCN: A Graph Memory Network for Cross-Sequence Pattern Reuse in 3D Human Pose Estimation

TL;DR

PRGCN addresses the ill-posed nature of monocular 3D human pose estimation by introducing a graph memory bank that stores learned pose prototypes and retrieves them as structured priors to guide 3D reconstruction. A memory-driven graph convolution adaptively fuses these retrieved patterns with static anatomical constraints, ensuring geometric plausibility. The architecture combines a local-temporal Mamba stream with a global self-attention stream, enabling robust pattern retrieval and efficient computation. Empirical results on Human3.6M and MPI-INF-3DHP establish state-of-the-art accuracy and demonstrate improved cross-domain generalization, validating cross-sequence knowledge reuse as a powerful paradigm for 3D pose estimation.

Abstract

Monocular 3D human pose estimation remains a fundamentally ill-posed inverse problem due to the inherent depth ambiguity in 2D-to-3D lifting. While contemporary video-based methods leverage temporal context to enhance spatial reasoning, they operate under a critical paradigm limitation: processing each sequence in isolation, thereby failing to exploit the strong structural regularities and repetitive motion patterns that pervade human movement across sequences. This work introduces the Pattern Reuse Graph Convolutional Network (PRGCN), a novel framework that formalizes pose estimation as a problem of pattern retrieval and adaptation. At its core, PRGCN features a graph memory bank that learns and stores a compact set of pose prototypes, encoded as relational graphs, which are dynamically retrieved via an attention mechanism to provide structured priors. These priors are adaptively fused with hard-coded anatomical constraints through a memory-driven graph convolution, ensuring geometrical plausibility. To underpin this retrieval process with robust spatiotemporal features, we design a dual-stream hybrid architecture that synergistically combines the linear-complexity, local temporal modeling of Mamba-based state-space models with the global relational capacity of self-attention. Extensive evaluations on Human3.6M and MPI-INF-3DHP benchmarks demonstrate that PRGCN establishes a new state-of-the-art, achieving an MPJPE of 37.1mm and 13.4mm, respectively, while exhibiting enhanced cross-domain generalization capability. Our work posits that the long-overlooked mechanism of cross-sequence pattern reuse is pivotal to advancing the field, shifting the paradigm from per-sequence optimization towards cumulative knowledge learning.

Figures (7)

• Figure 1: Conceptual illustration of the Pattern Reuse Graph Convolutional Network (PRGCN). The model processes 2D pose sequences (left) from the Human3.6M dataset and retrieves relevant pose prototypes from the central Graph Memory Bank. This allows for the reconstruction of geometrically plausible 3D poses (right), where joint coloring visualizes the retrieved structural patterns. This mechanism enables our model to leverage cross-sequence knowledge, forming a structured prior for estimation.
• Figure 2: Overview of the PRGCN architecture. The framework processes 2D pose sequences through three synergistic components: (a) Pattern retrieval identifies relevant pose prototypes from the graph memory bank; (b) A dual-stream architecture balances local and global modeling; (c) Memory-driven graph convolution fuses retrieved patterns with anatomical constraints. Red connections indicate temporal propagation of memory states.
• Figure 3: Qualitative results on the Human3.6M dataset. Using Stacked Hourglass detections.
• Figure 4: Cross-domain generalization evaluation: Zero-shot performance (P-MPJPE, mm) of models trained on Human3.6M and tested on MPI-INF-3DHP test sequences. TS1-TS6 denote the six test sequences, and Overall is the average. The value after $\pm$ is the standard deviation. The best result is in bold, and the second best is underlined. PRGCN achieves an overall error of 131.67mm, demonstrating the cross-domain transferability of learned pose prototypes.
• Figure 5: Qualitative results on the 3DPW dataset, demonstrating generalization ability in real-world in-the-wild scenarios.