Riemann$^2$: Learning Riemannian Submanifolds from Riemannian Data

TL;DR

Riemann$^2$ addresses the problem of learning low-dimensional representations for data constrained to Riemannian manifolds by endowing the latent space with a pullback metric induced by a manifold-aware decoder. It introduces a multitask Wrapped GPLVM to map latent codes to manifold-valued observations and derives a tractable, geometry-consistent framework for geodesics and distances in latent space. Key contributions include latent space geometries for multitask WGPLVMs, a derivation of the pullback metric distribution, and principled training with volume-change corrections and manifold-aware back constraints. The approach yields geometry-consistent trajectories in robotics, brain connectomes, and manipulability studies, demonstrating improved fidelity over Euclidean baselines and prior manifold-aware methods with reduced sensitivity to hyperparameters.

Abstract

Latent variable models are powerful tools for learning low-dimensional manifolds from high-dimensional data. However, when dealing with constrained data such as unit-norm vectors or symmetric positive-definite matrices, existing approaches ignore the underlying geometric constraints or fail to provide meaningful metrics in the latent space. To address these limitations, we propose to learn Riemannian latent representations of such geometric data. To do so, we estimate the pullback metric induced by a Wrapped Gaussian Process Latent Variable Model, which explicitly accounts for the data geometry. This enables us to define geometry-aware notions of distance and shortest paths in the latent space, while ensuring that our model only assigns probability mass to the data manifold. This generalizes previous work and allows us to handle complex tasks in various domains, including robot motion synthesis and analysis of brain connectomes.

Figures (8)

• Figure 1: Riemann$^2$ : To learn a Riemannian submanifold from Riemannian data, our method pulls back a Riemannian metric $\tilde{g}_{\bm{x}}$ to a latent space via a Wrapped GPLVM. In this model, each latent code $\bm{x}\in\mathbb{R}^L$ defines a distribution of tangent vectors $f_\text{E}(\bm{x})\sim \mathop{\mathrm{GP}}\nolimits(\bm{0}, k)$, which is then pushed forward onto the manifold $\mathcal{M}$ via the exponential map $\mathop{\mathrm{Exp}}\nolimits_{b(\cdot)}$. Our framework enables geodesics that, when decoded, comply with the data manifold and are guaranteed to lie on $\mathcal{M}$.
• Figure 2: Illustrative example on $\mathbb{R}^2 \times \mathbb{S}^2$: From left to right: Latent variables () with model uncertainty and magnification factor of the pullback metrics, demonstrations () and decoded geodesics (, ) on $\mathbb{R}^2$ and $\mathbb{S}^2$.
• Figure 3: $\mathbb{R}^3 \times \mathbb{S}^3$: Left: Latent variables () with magnification factor of the pullback metrics. One geodesic is depicted per model in the corresponding latent space. Right: Demonstrations () and reconstructions represented as positions and rotation frames. Rotations are not depicted for GPLVM and pGPLVM as their reconstructions do not lie on $\mathbb{S}^3$.
• Figure 4: $\mathbb{R}^3 \times \mathbb{S}^3$: Robot motions generated from the decoded WGPLVM and Riemann$^2$ geodesics. End-effector trajectories are represented as position and rotation frames.
• Figure 5: $\mathbb{R}^2 \times \mathcal{S}_{{{\mathcal{++}}}}^2$: From left to right: Latent variables () with magnification factor of the pullback metrics, demonstrations (,) and reconstructions depicted as curves and ellipsoids in $\mathbb{R}^2$, on the manifold $\mathcal{S}_{{{\mathcal{++}}}}^2$, and as ellipsoids over time.