Shape-aware Inertial Poser: Motion Tracking for Humans with Diverse Shapes Using Sparse Inertial Sensors

TL;DR

This work tackles the challenge of accurate sparse-IMU motion capture across diverse body shapes, emphasizing that IMU signals are shape-conditioned and cannot be modeled by pose alone. It introduces Shape-aware Inertial Poser (SAIP), a two-branch framework that first retargets shape-conditioned IMU signals to a template mean shape and then retrofits the motion back to the real body using a velocity retargeter, augmented by a shape-aware physics optimization. A novel shape-estimation module (MLP) and a Multi-shhape Inertial MoCap Dataset (MID) enable explicit shape awareness and training on a spectrum from children to adults. Experimental results show SAIP outperforms state-of-the-art methods on both near-template and shape-diverse subjects, with strong ablations confirming the contribution of shape conditioning, retargeting networks, and the physics-informed dynamic model. This approach advances practical, real-time inertial MoCap for heterogeneous populations and provides a valuable dataset for further research in shape-aware motion capture.

Abstract

Human motion capture with sparse inertial sensors has gained significant attention recently. However, existing methods almost exclusively rely on a template adult body shape to model the training data, which poses challenges when generalizing to individuals with largely different body shapes (such as a child). This is primarily due to the variation in IMU-measured acceleration caused by changes in body shape. To fill this gap, we propose Shape-aware Inertial Poser (SAIP), the first solution considering body shape differences in sparse inertial-based motion capture. Specifically, we decompose the sensor measurements related to shape and pose in order to effectively model their joint correlations. Firstly, we train a regression model to transfer the IMU-measured accelerations of a real body to match the template adult body model, compensating for the shape-related sensor measurements. Then, we can easily follow the state-of-the-art methods to estimate the full body motions of the template-shaped body. Finally, we utilize a second regression model to map the joint velocities back to the real body, combined with a shape-aware physical optimization strategy to calculate global motions on the subject. Furthermore, our method relies on body shape awareness, introducing the first inertial shape estimation scheme. This is accomplished by modeling the shape-conditioned IMU-pose correlation using an MLP-based network. To validate the effectiveness of SAIP, we also present the first IMU motion capture dataset containing individuals of different body sizes. This dataset features 10 children and 10 adults, with heights ranging from 110 cm to 190 cm, and a total of 400 minutes of paired IMU-Motion samples. Extensive experimental results demonstrate that SAIP can effectively handle motion capture tasks for diverse body shapes. The code and dataset are available at https://github.com/yinlu5942/SAIP.

Figures (8)

• Figure 1: Left: Live comparison on a child subject between the state-of-the-art inertial motion capture system PNP yi2024pnp (red) and our method (green). Our solution effectively handles the child character, whereas PNP shows errors. Right: Additional live demonstrations of our method showcase its capability to handle complex motions across diverse body shapes.
• Figure 2: Illustration of Our Motivation: IMU signals in real-life actions are influenced by various shape-related factors. Without kinematic signal retargeting, current baseline methods, trained solely on adult data, fail to accurately perform inertial motion tracking for subjects with diverse shapes, as the input signals deviate significantly from the training samples.
• Figure 3: Pipeline of Our Method: 1) Shape-Aware Motion Tracking: Real IMU signals are first mapped to a template-shaped model using $R_{acc}$. Next, a global motion estimator regresses local pose and joint velocities, followed by $R_{vel}$ to map the estimated velocities back to the target character. Our shape-aware motion optimizer employs a dynamic model tailored to the real body shape to derive physically accurate global motion. 2) Inertial Mesh Reconstruction: We utilize an MLP-based shape estimator to achieve shape awareness by leveraging IMU signals and the estimated pose.
• Figure 4: Our MID dataset contains IMU-Motion paired samples collected from 20 subjects with diverse shapes.
• Figure 5: Comparison of translation drifting error. We plot the global position error accumulation curve with respect to the real traveled distance. A lower curve indicates smaller drift. Left: Ablation study on our MID dataset demonstrate the effectiveness of the proposed modules on handling subjects with diverse body shapes. Right: Comparison with state-of-the-art PNP on the TotalCapture dataset shows that our method achieves higher accuracy even on near-template subjects.