ImDy: Human Inverse Dynamics from Imitated Observations

TL;DR

This work tackles the scalability challenge of human inverse dynamics by leveraging motion imitation to collect a large-scale, fully annotated dataset, ImDy, containing over 150 hours of motion with full-body torques and GRFs. It introduces ImDyS, a data-driven solver that predicts internal dynamics and ground reaction forces from kinematic observations using a transformer-based encoder and a set of physics-informed losses, trained with a two-stage sim2real curriculum. Across simulated ImDy data and real-world benchmarks (GroundLink and AddBiomechanics), ImDyS demonstrates superior torque and GRF prediction accuracy and better generalization than baselines, highlighting the potential of imitation-derived dynamics as a scalable tool for motion analysis. The approach enables downstream applications in human motion understanding and biomechanics, offering a practical pathway toward data-driven, physics-consistent analysis without extensive laboratory setups.

Abstract

Inverse dynamics (ID), which aims at reproducing the driven torques from human kinematic observations, has been a critical tool for gait analysis. However, it is hindered from wider application to general motion due to its limited scalability. Conventional optimization-based ID requires expensive laboratory setups, restricting its availability. To alleviate this problem, we propose to exploit the recently progressive human motion imitation algorithms to learn human inverse dynamics in a data-driven manner. The key insight is that the human ID knowledge is implicitly possessed by motion imitators, though not directly applicable. In light of this, we devise an efficient data collection pipeline with state-of-the-art motion imitation algorithms and physics simulators, resulting in a large-scale human inverse dynamics benchmark as Imitated Dynamics (ImDy). ImDy contains over 150 hours of motion with joint torque and full-body ground reaction force data. With ImDy, we train a data-driven human inverse dynamics solver ImDyS(olver) in a fully supervised manner, which conducts ID and ground reaction force estimation simultaneously. Experiments on ImDy and real-world data demonstrate the impressive competency of ImDyS in human inverse dynamics and ground reaction force estimation. Moreover, the potential of ImDy(-S) as a fundamental motion analysis tool is exhibited with downstream applications. The project page is https://foruck.github.io/ImDy/.

Figures (17)

• Figure 1: ImDy pairs diverse SMPL motion data with dynamics including full-body torques and ground reaction forces (GRF) like the right knee GRF for kneeling, which could be hard to achieve under conventional laboratory setups.
• Figure 2: ImDy construction. We first train a motion imitation policy following luo2023perpetual. Then, the policy is adopted to imitate arbitrary motions, with the imitated states recorded as ImDy.
• Figure 3: ImDyS overview. Taking a motion transition, ImDyS predicts the internal dynamics and ground reaction forces. Moreover, a prior discriminator is trained with the feature from ImDyS. A two-stage sim2real training curriculum is further designed.
• Figure 4: Qualitative results on ImDy. $\tilde{ \ }$ indicates a low-pass filter at 14Hz is applied. A typical gait sample and an arm-waving sample are visualized.
• Figure 5: Qualitative results on GroundLink including PHC, GroundLinkNet, and ImDyS. The GRF $\lambda$ for both feet are shown. Surprisingly, ImDyS provides better consistency with the ground truth.