Exploring Latent Cross-Channel Embedding for Accurate 3D Human Pose Reconstruction in a Diffusion Framework

TL;DR

This paper tackles depth ambiguity in monocular 3D human pose estimation by introducing a diffusion-model framework that jointly leverages 2D projections and 3D coordinates. The core innovations are the Cross-Channel Embedding module, which forms a rich 2D-3D feature coupling with an expanded cross-channel graph, and the Context Attention Guidance module, which propagates joint-level attention across latent diffusion channels. A Gaussian Mixture Model-based training pipeline replaces standard noise assumptions to better reflect realistic pose distributions. Empirical results on Human3.6M and MPI-INF-3DHP show state-of-the-art reconstruction accuracy, with ablations confirming the efficacy of the proposed components. The approach offers a practical, diffusion-based refinement stage that can enhance real-time monocular 3D HPE systems and can be extended to other diffusion-enabled pose estimation tasks.

Abstract

Monocular 3D human pose estimation poses significant challenges due to the inherent depth ambiguities that arise during the reprojection process from 2D to 3D. Conventional approaches that rely on estimating an over-fit projection matrix struggle to effectively address these challenges and often result in noisy outputs. Recent advancements in diffusion models have shown promise in incorporating structural priors to address reprojection ambiguities. However, there is still ample room for improvement as these methods often overlook the exploration of correlation between the 2D and 3D joint-level features. In this study, we propose a novel cross-channel embedding framework that aims to fully explore the correlation between joint-level features of 3D coordinates and their 2D projections. In addition, we introduce a context guidance mechanism to facilitate the propagation of joint graph attention across latent channels during the iterative diffusion process. To evaluate the effectiveness of our proposed method, we conduct experiments on two benchmark datasets, namely Human3.6M and MPI-INF-3DHP. Our results demonstrate a significant improvement in terms of reconstruction accuracy compared to state-of-the-art methods. The code for our method will be made available online for further reference.

Figures (3)

• Figure 1: System diagram for the proposed framework during inference. The distribution of the 2D and initial 3D pose predictions are fitted using a Gaussian Mixture Model, based on which $h_K= \{\mathbf{p}_K, \mathbf{d}_K\}$ will be sampled and go through $K$ iterations of reverse diffusion process until the high quality 3D pose $\hat{\mathbf{d}}_0$ is predicted.
• Figure 2: Illustration of information transfer path between node (mid-hip) and its neighbour in a GCN. Each curve demonstrates the pathway. Left: Cross-joint connection mentioned in gong2023diffposechoi2022diffupose. They only consider the cross-joint correlation. Right: Cross-channel connection generated by the proposed CCE module. Ours investigates the relations between 3D coordinates and 2D projections (highlighted by red curves).
• Figure 3: Qualitative evaluations of the proposed model trained on 2D ground truth. We provide two visual examples including the input 2D ground truth projection, 3D prediction and 3D ground truth (black) overlapped by 3D prediction.