Homogeneous Dynamics Space for Heterogeneous Humans

TL;DR

The paper tackles the fragmented landscape of human dynamics by addressing representation and domain heterogeneity across kinematics and dynamics. It introduces the Homogeneous Dynamics Space (HDyS), a framework that aggregates heterogeneous data and learns a unified latent space through inverse-forward dynamics auto-encoders (IDAE and FDAE) with reconstruction and alignment losses. HDyS demonstrates improved inverse dynamics accuracy and benefits downstream tasks such as GRF estimation, biomechanical simulation, and character control, illustrating the practical value of unified motor knowledge. This work provides a scalable approach to leverage diverse datasets (biomechanics, RL-based simulations, and large MoCap corpora) to enhance our understanding and application of human dynamics across domains.

Abstract

Analyses of human motion kinematics have achieved tremendous advances. However, the production mechanism, known as human dynamics, is still undercovered. In this paper, we aim to push data-driven human dynamics understanding forward. We identify a major obstacle to this as the heterogeneity of existing human motion understanding efforts. Specifically, heterogeneity exists in not only the diverse kinematics representations and hierarchical dynamics representations but also in the data from different domains, namely biomechanics and reinforcement learning. With an in-depth analysis of the existing heterogeneity, we propose to emphasize the beneath homogeneity: all of them represent the homogeneous fact of human motion, though from different perspectives. Given this, we propose Homogeneous Dynamics Space (HDyS) as a fundamental space for human dynamics by aggregating heterogeneous data and training a homogeneous latent space with inspiration from the inverse-forward dynamics procedure. Leveraging the heterogeneous representations and datasets, HDyS achieves decent mapping between human kinematics and dynamics. We demonstrate the feasibility of HDyS with extensive experiments and applications. The project page is https://foruck.github.io/HDyS.

Figures (7)

• Figure 1: The representation heterogeneity exists for both kinematics and dynamics.
• Figure 2: The overall architecture of HDyS.
• Figure 3: Feature visualization for HDyS w/ and w/o $L_{align}$.
• Figure 4: Inverse dynamics visualization on ImDy.
• Figure 5: Inverse dynamics visualization on AddBiomechanics.