Improving Global Motion Estimation in Sparse IMU-based Motion Capture with Physics

TL;DR

This paper tackles the challenge of global motion estimation in sparse IMU-based motion capture by introducing a physics-informed framework that integrates gravity-aware pose estimation, a gravity-conditioned translation model, and a 3D-space physics optimizer. A double-tracking approach discovers 3D contacts and refines motion using a torque-controlled floating-base physics character, yielding physically plausible translations and orientations from only 6 IMUs. Key contributions include gravity-directed pose refinement, 3D contact estimation with friction-cone constraints, and the ability to estimate byproducts such as 3D contact forces and joint torques, all operating in real time. The method demonstrates improved pose accuracy and reduced translation drift across datasets, with strong performance on unconstrained, long-duration sequences and practical walking-calibration advantages for real-world deployment.

Abstract

By learning human motion priors, motion capture can be achieved by 6 inertial measurement units (IMUs) in recent years with the development of deep learning techniques, even though the sensor inputs are sparse and noisy. However, human global motions are still challenging to be reconstructed by IMUs. This paper aims to solve this problem by involving physics. It proposes a physical optimization scheme based on multiple contacts to enable physically plausible translation estimation in the full 3D space where the z-directional motion is usually challenging for previous works. It also considers gravity in local pose estimation which well constrains human global orientations and refines local pose estimation in a joint estimation manner. Experiments demonstrate that our method achieves more accurate motion capture for both local poses and global motions. Furthermore, by deeply integrating physics, we can also estimate 3D contact, contact forces, joint torques, and interacting proxy surfaces.

Figures (7)

• Figure 1: Illustration of the correlation between human local pose and global orientation. Given a local pose, the global $\phi$ orientation of the character is strongly constrained, while the heading direction $\theta$ can vary.
• Figure 2: Overview of our method. We begin by estimating the human pose from IMU measurements (red). During this process, we simultaneously refine the root-relative gravity direction, which aids both local and global pose estimation. Next, we estimate human root velocity and joint stationary probability based on the pose and IMU measurements (blue). To incorporate gravity awareness, we decompose the root velocity into orthogonal components parallel and perpendicular to the gravity direction. Finally, we identify 3D contacts from the stationary joints using a physics-based algorithm, and perform physics optimization on the estimated motion (green).
• Figure 3: Translation comparisons on the TotalCapture dataset. We plot the cumulative translation error curves and report the average translation drifts at the 7-meter real travelled distance. A lower curve indicates better global translation accuracy.
• Figure 4: Qualitative comparisons on full pose estimation (including both local pose and global orientation). Results are picked from the TotalCapture dataset.
• Figure 5: Voting results comparing walking-based calibration and T-pose calibration across 100 evaluations. Walking-based calibration was preferred in 72% of cases, T-pose calibration in 12%, and 16% were rated as comparable.