Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

(Deep) Generative Geodesics

Beomsu Kim, Michael Puthawala, Jong Chul Ye, Emanuele Sansone

TL;DR

The paper tackles global geometric understanding of generative models by introducing a model-agnostic Riemannian metric $d_{\Psi,\lambda}$ derived from the data likelihood $p_\Psi$ and a tunable parameter $\lambda$. It formalizes a metric via $g_{x,\Psi,\lambda}$ and defines generative geodesics, while proving convergence of graph-based discrete approximations to the true geodesics as data grow and discretization improves, using an $\epsilon$-graph with quadrature-based edge costs. The authors demonstrate the utility of this geometry through three applications: clustering, low-dimensional embedding, and interpolation, highlighting robustness and improved structural insights over Euclidean baselines. This work provides a computationally tractable framework to analyze and exploit the global geometry induced by generative models, independent of their internal parametrization. The approach combines Riemannian geometry with graph-theoretic approximations to yield practical tools for understanding data manifolds shaped by generative processes.

Abstract

In this work, we propose to study the global geometrical properties of generative models. We introduce a new Riemannian metric to assess the similarity between any two data points. Importantly, our metric is agnostic to the parametrization of the generative model and requires only the evaluation of its data likelihood. Moreover, the metric leads to the conceptual definition of generative distances and generative geodesics, whose computation can be done efficiently in the data space. Their approximations are proven to converge to their true values under mild conditions. We showcase three proof-of-concept applications of this global metric, including clustering, data visualization, and data interpolation, thus providing new tools to support the geometrical understanding of generative models.

(Deep) Generative Geodesics

TL;DR

The paper tackles global geometric understanding of generative models by introducing a model-agnostic Riemannian metric derived from the data likelihood and a tunable parameter . It formalizes a metric via and defines generative geodesics, while proving convergence of graph-based discrete approximations to the true geodesics as data grow and discretization improves, using an -graph with quadrature-based edge costs. The authors demonstrate the utility of this geometry through three applications: clustering, low-dimensional embedding, and interpolation, highlighting robustness and improved structural insights over Euclidean baselines. This work provides a computationally tractable framework to analyze and exploit the global geometry induced by generative models, independent of their internal parametrization. The approach combines Riemannian geometry with graph-theoretic approximations to yield practical tools for understanding data manifolds shaped by generative processes.

Abstract

In this work, we propose to study the global geometrical properties of generative models. We introduce a new Riemannian metric to assess the similarity between any two data points. Importantly, our metric is agnostic to the parametrization of the generative model and requires only the evaluation of its data likelihood. Moreover, the metric leads to the conceptual definition of generative distances and generative geodesics, whose computation can be done efficiently in the data space. Their approximations are proven to converge to their true values under mild conditions. We showcase three proof-of-concept applications of this global metric, including clustering, data visualization, and data interpolation, thus providing new tools to support the geometrical understanding of generative models.
Paper Structure (17 sections, 5 theorems, 16 equations, 9 figures, 1 algorithm)

This paper contains 17 sections, 5 theorems, 16 equations, 9 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background on Geodesics and Metrics
  3. Our Contribution
  4. Generative Geodesics
  5. Approximation and Convergence Guarantee
  6. Applications/Experiments
  7. Clustering with Geodesics
  8. Embedding with Geodesics
  9. Interpolating with Geodesics
  10. Proofs
  11. Proof of Theorem \ref{['thm:conv-of-k-approx-lin-interp-cost-to-geodesic-cost']}
  12. Algorithm
  13. Experiment Settings
  14. Section \ref{['sec:deepgenerative:approx-and-conv']}
  15. Section \ref{['sec:experiments_clustering']}
  16. ...and 2 more sections

Key Result

Lemma 2.1

Let $\lambda, p_0 > 0$, and $p_\Psi$ be a smooth probability density on $\mathbb{R}^n$. Then $d_{\Psi,\lambda}$ is a Riemannian metric on $\mathbb{R}^n$. Further, for $x,y \in \mathbb{R}^n$, Moreover, if $\Omega$ is bounded, then this convergence is uniform (i.e. doesn't depend on $x,y$).

Figures (9)

  • Figure 1: A visualization of how different paths can be optimal under different preferences. The blue path has the least walking, the yellow path is the most scenic, and the red path has the least sun exposure. Image generated with Stable Diffusion model.
  • Figure 2: A showcase of how decreasing $\lambda$ changes the shortest path between the two red points. The black curve has $\lambda = 5$, the white curve $\lambda = 0.05$, and the other curves interpolate between the two. The background $p_\Psi(x)$ probability distribution is shown with a yellow to blue gradient with yellow indicating high probability and blue low probability.
  • Figure 3: A figure demonstrating the formation of the graph $\mathcal{G}_\epsilon(\mathcal{X})$, for different choices of $\epsilon$. In both figures, the black dots denote points in $\mathcal{X}$, and black lines denote the edges in $E_\epsilon$. Notice how when $\epsilon = 0.56$, as in \ref{['fig:eps-graph:2']}, the $\mathcal{G}$ has one connected component, whereas when $\epsilon = 0.4$, as in \ref{['fig:eps-graph:3']}, $\mathcal{G}$ has several disconnected components.
  • Figure 4: Error analysis of the true geodesic length vs the length of the approximated geodesic as described in Theorem \ref{['thm:conv-of-k-approx-lin-interp-cost-to-geodesic-cost']}, in a log plot. The $x$ axis indicates the number of points and the $y$ axis indicates the absolute error in the geodesic approximation. The blue line uses points sampled uniformly from the ambient space, and the orange line uses points sampled from $p_\Psi$.
  • Figure 5: Euclidean and geodesic affinity visualization on various toy datasets. First column shows data samples, where color denotes the cluster each sample was generated from. Second column shows data density, where brighter color means higher density. Third and fourth columns show Euclidean and geodesic affinity, respectively, between the red cross and rest of the points. Brighter color means higher affinity.
  • ...and 4 more figures

Theorems & Definitions (13)

  • Lemma 2.1: $d_{\Psi,\lambda}$ Metric
  • Definition 2.2: Generative Geodesic
  • Definition 2.3: $\epsilon$ Graph
  • Definition 2.4: Weighted $\epsilon$ graph
  • Theorem 2.5: Convergence of Linear Interpolation Costs to Riemannian Distance
  • Definition 1.1: Linear Interpolating Cost
  • Proposition 1.2: Convergence of Linear Interpolation Costs to Geodesic Cost
  • proof
  • Lemma 1.3: Approximating Edge Weights
  • proof
  • ...and 3 more