Taxonomy-aware Dynamic Motion Generation on Hyperbolic Manifolds

TL;DR

Experiments on generating realistic motion sequences on the hand grasping taxonomy show that the proposed GPHDM faithfully encodes the underlying taxonomy and temporal dynamics, and it generates novel physically-consistent trajectories.

Abstract

Human-like motion generation for robots often draws inspiration from biomechanical studies, which often categorize complex human motions into hierarchical taxonomies. While these taxonomies provide rich structural information about how movements relate to one another, this information is frequently overlooked in motion generation models, leading to a disconnect between the generated motions and their underlying hierarchical structure. This paper introduces the \ac{gphdm}, a novel approach that learns latent representations preserving both the hierarchical structure of motions and their temporal dynamics to ensure physical consistency. Our model achieves this by extending the dynamics prior of the Gaussian Process Dynamical Model (GPDM) to the hyperbolic manifold and integrating it with taxonomy-aware inductive biases. Building on this geometry- and taxonomy-aware frameworks, we propose three novel mechanisms for generating motions that are both taxonomically-structured and physically-consistent: two probabilistic recursive approaches and a method based on pullback-metric geodesics. Experiments on generating realistic motion sequences on the hand grasping taxonomy show that the proposed GPHDM faithfully encodes the underlying taxonomy and temporal dynamics, and it generates novel physically-consistent trajectories.

Figures (5)

• Figure 1: Embeddings of hand grasps colored according to the grasp class of the last trajectory point with colors matching those of Fig. \ref{['fig:teaser']}. The top and bottom rows show $2$- and $3$-dimensional latent spaces, respectively.
• Figure 2: Left: Embeddings of hand grasps and hyperbolic geodesics () from a lateral () to a stick () grasp. Right: Representative dimension of the probabilistic hand motion prediction for the geodesics () with mean and uncertainty, along with training trajectories for the lateral (reversed) () and stick () grasps. The top and bottom rows show $2$- and $3$-dimensional latent spaces.
• Figure 3: Latent trajectories () obtained via recursive motion generation in a 3-dimensional gphdm (zoomed-in). Left: Mean prediction towards a lateral grasp (). Right: Conditional prediction from an index finger extension () to a lateral () grasp.
• Figure 4: Illustration of the directionality induced by the gpdm's Markov prior on latent trajectories obtained via conditional predictions. The outward transition (left) from an index finger extension () to a lateral () grasp follows a training trajectory, while the reverse transition (right) avoids the training data.
• Figure 5: Top left: 3-dimensional embeddings of hand grasps via a gphdm with a hyperbolic () and a pullback () geodesic from a ring () to a spherical () grasp. Top right: Representative dimension of the probabilistic hand motion prediction with mean and uncertainty along with training trajectories for the spherical () and ring (reversed) () grasps. Bottom: Generated hand motions from the decoded geodesics.