Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bringing motion taxonomies to continuous domains via GPLVM on hyperbolic manifolds

Noémie Jaquier, Leonel Rozo, Miguel González-Duque, Viacheslav Borovitskiy, Tamim Asfour

TL;DR

The paper tackles converting discrete motion taxonomies into usable continuous representations for motion generation by introducing a Gaussian process latent variable model on hyperbolic manifolds (GPHLVM). It formulates hyperbolic GPs, a hyperbolic wrapped Gaussian prior, and Riemannian optimization, and integrates taxonomy structure via graph-distance priors and back-constraints. Empirical results on three taxonomy datasets show hyperbolic embeddings better preserve graph distances than Euclidean ones, enable robust unseen-pose encoding, and support geodesic-based motion generation with competitive realism at low data regimes. The approach offers taxonomy-aware, uncertainty-aware, and geometry-consistent embeddings with practical implications for robot motion planning, manipulation, and animation.

Abstract

Human motion taxonomies serve as high-level hierarchical abstractions that classify how humans move and interact with their environment. They have proven useful to analyse grasps, manipulation skills, and whole-body support poses. Despite substantial efforts devoted to design their hierarchy and underlying categories, their use remains limited. This may be attributed to the lack of computational models that fill the gap between the discrete hierarchical structure of the taxonomy and the high-dimensional heterogeneous data associated to its categories. To overcome this problem, we propose to model taxonomy data via hyperbolic embeddings that capture the associated hierarchical structure. We achieve this by formulating a novel Gaussian process hyperbolic latent variable model that incorporates the taxonomy structure through graph-based priors on the latent space and distance-preserving back constraints. We validate our model on three different human motion taxonomies to learn hyperbolic embeddings that faithfully preserve the original graph structure. We show that our model properly encodes unseen data from existing or new taxonomy categories, and outperforms its Euclidean and VAE-based counterparts. Finally, through proof-of-concept experiments, we show that our model may be used to generate realistic trajectories between the learned embeddings.

Bringing motion taxonomies to continuous domains via GPLVM on hyperbolic manifolds

TL;DR

The paper tackles converting discrete motion taxonomies into usable continuous representations for motion generation by introducing a Gaussian process latent variable model on hyperbolic manifolds (GPHLVM). It formulates hyperbolic GPs, a hyperbolic wrapped Gaussian prior, and Riemannian optimization, and integrates taxonomy structure via graph-distance priors and back-constraints. Empirical results on three taxonomy datasets show hyperbolic embeddings better preserve graph distances than Euclidean ones, enable robust unseen-pose encoding, and support geodesic-based motion generation with competitive realism at low data regimes. The approach offers taxonomy-aware, uncertainty-aware, and geometry-consistent embeddings with practical implications for robot motion planning, manipulation, and animation.

Abstract

Human motion taxonomies serve as high-level hierarchical abstractions that classify how humans move and interact with their environment. They have proven useful to analyse grasps, manipulation skills, and whole-body support poses. Despite substantial efforts devoted to design their hierarchy and underlying categories, their use remains limited. This may be attributed to the lack of computational models that fill the gap between the discrete hierarchical structure of the taxonomy and the high-dimensional heterogeneous data associated to its categories. To overcome this problem, we propose to model taxonomy data via hyperbolic embeddings that capture the associated hierarchical structure. We achieve this by formulating a novel Gaussian process hyperbolic latent variable model that incorporates the taxonomy structure through graph-based priors on the latent space and distance-preserving back constraints. We validate our model on three different human motion taxonomies to learn hyperbolic embeddings that faithfully preserve the original graph structure. We show that our model properly encodes unseen data from existing or new taxonomy categories, and outperforms its Euclidean and VAE-based counterparts. Finally, through proof-of-concept experiments, we show that our model may be used to generate realistic trajectories between the learned embeddings.
Paper Structure (52 sections, 1 theorem, 31 equations, 23 figures, 13 tables, 3 algorithms)

This paper contains 52 sections, 1 theorem, 31 equations, 23 figures, 13 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Gaussian Process Latent Variable Models:
  4. Riemannian geometry:
  5. Hyperbolic manifold:
  6. Hyperbolic wrapped Gaussian distribution:
  7. The proposed GPHLVM
  8. Hyperbolic kernels
  9. Model training
  10. Variational inference:
  11. Optimization:
  12. Incorporating Taxonomy Knowledge
  13. Graph-distance priors:
  14. Back-constraints:
  15. Experiments
  16. ...and 37 more sections

Key Result

Proposition 2.1

Assume the disk model of $\mathcal{P}^{2}$ (i.e. the Poincaré disk). Denote the disk by $\mathbb{D}$ and its boundary, the circle, by $\mathbb{T}$. Define the hyperbolic outer product by $\langle \bm{z}, \bm{b} \rangle = \frac{1}{2}\log\frac{1-|\bm{z}|^2}{|\bm{z}-\bm{b}|^2}$ for $\bm{z} \in \mathbb{ where $\bm{z} \in \mathbb{D}$ is such that $\rho = \operatorname{dist}_{\mathcal{P}^{2}}(\bm{z}, \b

Figures (23)

  • Figure 1: Left: Illustration of the Lorentz $\mathcal{L}^2$ and Poincaré $\mathcal{P}^2$ models of the hyperbolic manifold. The former is depicted as the gray hyperboloid, while the latter is represented by the blue circle. Both models show a geodesic () between two points $\bm{x}_1$ () and $\bm{x}_2$ (). The vector $\bm{u}$ () lies on the tangent space of $\bm{x}_1$ such that $\bm{u} = \text{Log}_{\bm{x}_1}(\bm{x}_2)$. Right: Hand grasp taxonomy Stival19:HumanGraspTaxonomy used in one of our experiments. Grasp types are organized in a tree structure based on their muscular and kinematic properties. Each leaf node of the tree is a hand grasp type. The lines represent the depth of the leaves, e.g., $\mathsf{PE}$ and $\mathsf{IE}$ are at distance $2$ and $3$ from the root node.
  • Figure 2: Bimanual manipulation categories: The first and last two rows show the latent embeddings and examples of interpolating geodesics in $\mathcal{P}^2$ and $\mathbb{R}^2$, followed by pairwise error matrices between geodesic and taxonomy graph distances. Background colors indicate the GPLVM uncertainty. Added poses (d) and classes $\mathsf{TCA}_{\text{right}}$(e) are marked with stars and highlighted with red in the error matrices.
  • Figure 3: Grasps: The first and last two rows show the latent embeddings and examples of interpolating geodesics in $\mathcal{P}^2$ and $\mathbb{R}^2$, followed by pairwise error matrices between geodesic and graph distances. Embeddings colors match those of Fig. \ref{['fig:HyperbolicAndTaxonomy']}-right, and background colors indicate the GPLVM uncertainty. Added poses (d) and classes $\mathsf{Qu}, \mathsf{St}, \mathsf{MW},$ and $\mathsf{Ri}$(e) are marked with stars and highlighted with red in the error matrices.
  • Figure 4: Motions obtained via geodesic interpolation in the back-constrained GPHLVM latent space. Top: Grasp taxonomy from ring ($\mathsf{Ri}$) to index finger extension ($\mathsf{IE}$). Bottom: Support pose taxonomy from $\mathsf{LF}\mathsf{RH}$ to $\mathsf{K}_2\mathsf{RH}$. Gray circles denote contacts.
  • Figure 5: Principal Riemannians operation on the Lorentz model $\mathcal{L}^{2}$. (a) The geodesic () is the shortest path between the two points $\bm{x}$ to $\bm{y}$ on the manifold. The vector $\bm{u}$ () lies on the tangent space of $\bm{x}$ such that $\bm{y} = \text{Exp}_{\bm{x}}(\bm{u})$. (b)$\mathrm{P}_{\bm{x} \rightarrow \bm{y}}(\bm{v})$ is the parallel transport of the vector $\bm{v}$ from $\mathcal{T}_{\bm{x}}\mathcal{L}^{2}$ to $\mathcal{T}_{\bm{y}}\mathcal{L}^{2}$.
  • ...and 18 more figures

Theorems & Definitions (2)

  • Proposition 2.1
  • proof