Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors

TL;DR

ProgIP tackles real-time full-body pose estimation using only three IMU sensors on the head and wrists, an under-constrained problem for which biomechanical constraints and temporal modeling are essential. It combines a TE-biLSTM encoder with an MLP decoder within a four-region progressive kinematic-chain framework, augmented by a global feature extractor and a forward-kinematics joint-position consistency loss to map inertial signals to SMPL pose parameters $\theta$. Across AMASS, DIP-IMU, and TotalCapture (with synthetic augmentation), ProgIP achieves state-of-the-art accuracy for 3-IMU inputs and approaches the performance of six-IMU methods, while delivering real-time operation at 60 Hz and a latency of about $166\mathrm{ms}$. This approach reduces hardware requirements for VR motion capture and provides a practical, robust solution for real-world full-body tracking in challenging environments.

Abstract

The motion capture system that supports full-body virtual representation is of key significance for virtual reality. Compared to vision-based systems, full-body pose estimation from sparse tracking signals is not limited by environmental conditions or recording range. However, previous works either face the challenge of wearing additional sensors on the pelvis and lower-body or rely on external visual sensors to obtain global positions of key joints. To improve the practicality of the technology for virtual reality applications, we estimate full-body poses using only inertial data obtained from three Inertial Measurement Unit (IMU) sensors worn on the head and wrists, thereby reducing the complexity of the hardware system. In this work, we propose a method called Progressive Inertial Poser (ProgIP) for human pose estimation, which combines neural network estimation with a human dynamics model, considers the hierarchical structure of the kinematic chain, and employs a multi-stage progressive network estimation with increased depth to reconstruct full-body motion in real time. The encoder combines Transformer Encoder and bidirectional LSTM (TE-biLSTM) to flexibly capture the temporal dependencies of the inertial sequence, while the decoder based on multi-layer perceptrons (MLPs) transforms high-dimensional features and accurately projects them onto Skinned Multi-Person Linear (SMPL) model parameters. Quantitative and qualitative experimental results on multiple public datasets show that our method outperforms state-of-the-art methods with the same inputs, and is comparable to recent works using six IMU sensors.

Figures (9)

• Figure 1: The pipeline of our method. We divide the human body into four regions based on the hierarchical structure of the kinematic chain and use multi-stage progressive pose estimation to achieve real-time full-body motion synthesis. First, the full-body motion information ${\bm{p}_{global}}$ is roughly estimated from the IMU measurements, and its output is combined with the IMU measurements $\bm{X}^{(0)}$ as $\bm{X}^{(1)}$. The progressive estimation process is divided into four stages: (1) The first stage estimates joint poses in the first region from input $\bm{X}^{(1)}$, and the output $\left[ {\bm{p}_{1}},\bm{p}_{{\text{pelvis}}}^{(1)} \right]$ is concatenated with $\bm{X}^{(1)}$ to form $\bm{X}^{(2)}$; (2) The second stage estimates joint poses in the second region from input $\bm{X}^{(2)}$, and output $\left[ {\bm{p}_{2}},\bm{p}_{{\text{pelvis}}}^{(2)} \right]$ is concatenated with $\bm{X}^{(1)}$ to form $\bm{X}^{(3)}$; (3) The third stage estimates joint poses in the third region from input $\bm{X}^{(3)}$, and outputs $\left[ {\bm{p}_{3}},\bm{p}_{{\text{pelvis}}}^{(3)} \right]$ represents the pose of the upper-body joints including the pelvis; (4) The fourth stage estimates joint poses in the fourth region from concatenated input of $\bm{X}^{(1)}$ and $\bm{p}_{{\text{pelvis}}}^{(1)}$, and outputs $\bm{p}_{4}$ represents lower-body poses. Finally, we combine $\left[ {\bm{p}_{\text{3}}},\bm{p}_{{\text{pelvis}}}^{(3)} \right]$ and $\bm{p}_{\text{4}}$ to obtain full-body poses and project them onto the SMPL model.
• Figure 2: The proposed ProgIP generates full-body poses by using only acceleration and rotation data from the head and wrists. The left image illustrates the IMU placement, where the sensors are tightly bound with arbitrary orientations.
• Figure 3: The detailed structure of the backbone network in the pipeline. It mainly includes the TE-biLSTM encoder and the MLP-based decoder, and the final two decoders in the network output the pose of the pelvic joint and the poses of the other joints, respectively.
• Figure 4: Human body region division and hierarchical structure in the dynamic model. (a) We modify the human body region division from previous work by dividing the human body into four regions, with specific attention given to the neck, left collar, and right collar as a transition region. (b) We present the joint poses of the four divided body regions in detail in order of kinematic chain depth, gradually increasing from the pelvis to the upper and lower body.
• Figure 5: The mean position error of the full-body joints along the x-axis, y-axis, and z-axis of the partial sequence in the TotalCapture dataset. The blue line represents the mean estimated joint position, the orange line represents the mean ground truth joint position, and the green line represents the average position error.