Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HDiffTG: A Lightweight Hybrid Diffusion-Transformer-GCN Architecture for 3D Human Pose Estimation

Yajie Fu, Chaorui Huang, Junwei Li, Hui Kong, Yibin Tian, Huakang Li, Zhiyuan Zhang

TL;DR

HDiffTG addresses 3D human pose estimation from monocular images by integrating a lightweight Transformer-GCN dual-stream backbone with a diffusion-based refinement stage. The approach uses adaptive fusion to combine global spatiotemporal cues with local skeletal structure, and introduces a PDE-inspired smoothing mechanism along with an accelerated diffusion objective to reduce computational cost. An improved DDIM-based sampling and embedding-dimension adjustments enable high-quality pose estimates with fewer diffusion steps, delivering robustness to occlusion and 2D keypoint noise. Evaluations on Human3.6M and MPI-INF-3DHP show competitive or state-of-the-art accuracy with superior efficiency, highlighting practical potential for real-world 3D pose estimation tasks.

Abstract

We propose HDiffTG, a novel 3D Human Pose Estimation (3DHPE) method that integrates Transformer, Graph Convolutional Network (GCN), and diffusion model into a unified framework. HDiffTG leverages the strengths of these techniques to significantly improve pose estimation accuracy and robustness while maintaining a lightweight design. The Transformer captures global spatiotemporal dependencies, the GCN models local skeletal structures, and the diffusion model provides step-by-step optimization for fine-tuning, achieving a complementary balance between global and local features. This integration enhances the model's ability to handle pose estimation under occlusions and in complex scenarios. Furthermore, we introduce lightweight optimizations to the integrated model and refine the objective function design to reduce computational overhead without compromising performance. Evaluation results on the Human3.6M and MPI-INF-3DHP datasets demonstrate that HDiffTG achieves state-of-the-art (SOTA) performance on the MPI-INF-3DHP dataset while excelling in both accuracy and computational efficiency. Additionally, the model exhibits exceptional robustness in noisy and occluded environments. Source codes and models are available at https://github.com/CirceJie/HDiffTG

HDiffTG: A Lightweight Hybrid Diffusion-Transformer-GCN Architecture for 3D Human Pose Estimation

TL;DR

HDiffTG addresses 3D human pose estimation from monocular images by integrating a lightweight Transformer-GCN dual-stream backbone with a diffusion-based refinement stage. The approach uses adaptive fusion to combine global spatiotemporal cues with local skeletal structure, and introduces a PDE-inspired smoothing mechanism along with an accelerated diffusion objective to reduce computational cost. An improved DDIM-based sampling and embedding-dimension adjustments enable high-quality pose estimates with fewer diffusion steps, delivering robustness to occlusion and 2D keypoint noise. Evaluations on Human3.6M and MPI-INF-3DHP show competitive or state-of-the-art accuracy with superior efficiency, highlighting practical potential for real-world 3D pose estimation tasks.

Abstract

We propose HDiffTG, a novel 3D Human Pose Estimation (3DHPE) method that integrates Transformer, Graph Convolutional Network (GCN), and diffusion model into a unified framework. HDiffTG leverages the strengths of these techniques to significantly improve pose estimation accuracy and robustness while maintaining a lightweight design. The Transformer captures global spatiotemporal dependencies, the GCN models local skeletal structures, and the diffusion model provides step-by-step optimization for fine-tuning, achieving a complementary balance between global and local features. This integration enhances the model's ability to handle pose estimation under occlusions and in complex scenarios. Furthermore, we introduce lightweight optimizations to the integrated model and refine the objective function design to reduce computational overhead without compromising performance. Evaluation results on the Human3.6M and MPI-INF-3DHP datasets demonstrate that HDiffTG achieves state-of-the-art (SOTA) performance on the MPI-INF-3DHP dataset while excelling in both accuracy and computational efficiency. Additionally, the model exhibits exceptional robustness in noisy and occluded environments. Source codes and models are available at https://github.com/CirceJie/HDiffTG
Paper Structure (15 sections, 11 equations, 4 figures, 6 tables)

This paper contains 15 sections, 11 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Transformer based methods
  4. Graph Convolutional Network (GCN)-based methods
  5. Diffusion based methods
  6. Method
  7. Transformer-GCN Dual-Stream Module
  8. Diffusion Based Refinement
  9. Experiments
  10. Datasets and Metrics
  11. Experimental Setup
  12. Results and Analysis
  13. Ablation Studies
  14. Conclusion
  15. Acknowledgment

Figures (4)

  • Figure 1: Comparison of different 3D human pose estimation methods on the Human3.6M dataset in terms of Param and estimation error (MPJPE, lower is better). Our approach achieves competitive performance metrics while maintaining a lightweight model.
  • Figure 2: The architecture of HDiffTG. HDiffTG consists of N parallel dual-stream fusion modules combining Transformers and GCNs. The spatial stream processes individual human joints (17 in total), while the temporal stream operates on whole-body poses across frames.
  • Figure 3: The performance variations of HDiffTG, D3DP, FinePose, and DDHPose on the MPI-INF-3DHP dataset when Gaussian noise with a mean of 0 and standard deviation $(\sigma)$ is added are analyzed.
  • Figure 4: Qualitative comparison of our HDiffTG with the state of the art 3D pose estimation approach, MotionBert MotionBERT on Human3.6M under noise and joint occlusion.