Latent Space Non-Linear Statistics

TL;DR

The paper addresses the challenge that standard Euclidean statistics do not apply to latent spaces learned by deep generative models, where the latent manifold inherits a nonlinear Riemannian geometry from the data space. It leverages nonlinear manifold statistics—Fréchet/ML means, principal geodesic analysis (PGA), and diffusion-based inference—to perform meaningful statistics directly in the latent space, while introducing a neural network to rapidly approximate the latent metric g and its cometric g^{-1}. Key contributions include coupling nonlinear statistics with pre-trained latent representations, a fast metric-approximation network that dramatically speeds up geometric computations, and diffusion-bridge–based maximum likelihood inference for latent means, demonstrated on synthetic data and MNIST within the Theano Geometry framework. The work provides geometry-aware, sample-efficient tools for analyzing data on learned manifolds, with potential impact in domains such as medical imaging where labeled data are scarce but unlabeled data are abundant.

Abstract

Given data, deep generative models, such as variational autoencoders (VAE) and generative adversarial networks (GAN), train a lower dimensional latent representation of the data space. The linear Euclidean geometry of data space pulls back to a nonlinear Riemannian geometry on the latent space. The latent space thus provides a low-dimensional nonlinear representation of data and classical linear statistical techniques are no longer applicable. In this paper we show how statistics of data in their latent space representation can be performed using techniques from the field of nonlinear manifold statistics. Nonlinear manifold statistics provide generalizations of Euclidean statistical notions including means, principal component analysis, and maximum likelihood fits of parametric probability distributions. We develop new techniques for maximum likelihood inference in latent space, and adress the computational complexity of using geometric algorithms with high-dimensional data by training a separate neural network to approximate the Riemannian metric and cometric tensor capturing the shape of the learned data manifold.

Figures (6)

• Figure 1: (left) Samples from the data distribution (blue) with corresponding predictions from the VAE (red). (right) The trained manifold.
• Figure 2: (left) Brownian bridge sample paths on the trained data manifold. (middle) The estimated ML mean (blue) from the data (black points). (right) The likelihood values from the MLE procedure.
• Figure 3: (left) Scalar curvature of space $Z$. (middle) Min. eigenvalue of the Ricci curvature tensor. (right) Parallel transport of a tangent vector in $Z$. The transported vectors have constant length measured by the Riemannian metric.
• Figure 4: (top left) Samples from a Riemannian Brownian motion in latent space. (top right) Samples from a Brownian bridge simulated by \ref{['guided_proc']}. (bottom) Examples of Brownian bridges of MNIST data between two fixed 9s (left-/rightmost). The variance of the Brownian motion has been increased to visually emphasize the image variation.
• Figure 5: (top left) Latent space representation of data. (top right) PGA analysis on the sub-space of even digits. (bottom) Variation along first (1. row) and second (2. row) principal components.