Likely Interpolants of Generative Models

TL;DR

Interpolation in generative models is reframed as finding curves on a Riemannian manifold constrained by data density. The authors introduce ProbGEORCE, a training-free method that minimizes a regularized energy $E(z)=\sum_i ((z_{i+1}-z_i)^T G(z_i)(z_{i+1}-z_i) + \lambda S(z_i))$ with endpoints $z_0=a$, $z_N=b$, and prove convergence to a local minimum while revealing a local geodesic interpretation via a metric $G(z_i^*) + \frac{\lambda}{2} \partial^2_{zz} S(z_i^*)$. Empirically, the approach yields interpolations that traverse higher-density regions across diffusion models, Riemannian diffusion models, and VAEs, with improved log-likelihoods and competitive FID relative to baselines. This density-aware, model-agnostic interpolation framework enables high-quality, transition-rich visualizations and inspections without retraining, with potential impact on controlled generation and model introspection.

Abstract

Interpolation in generative models allows for controlled generation, model inspection, and more. Unfortunately, most generative models lack a principal notion of interpolants without restrictive assumptions on either the model or data dimension. In this paper, we develop a general interpolation scheme that targets likely transition paths compatible with different metrics and probability distributions. We consider interpolants analogous to a geodesic constrained to a suitable data distribution and derive a novel algorithm for computing these curves, which requires no additional training. Theoretically, we show that our method locally can be considered as a geodesic under a suitable Riemannian metric. We quantitatively show that our interpolation scheme traverses higher density regions than baselines across a range of models and datasets.

Figures (12)

• Figure 1: Conceptual illustration of our method: we consider interpolation in generative models as computing curves (black) on a Riemannian manifold constrained to the data distribution (colored).
• Figure 2: Interpolation using a latent diffusion model similar to pinaya2022brainimaginggenerationlatent with our proposed method. Each row represents different slices of the brain. The interpolation shows the generative transition between a healthy brain (left) and a brain with Alzheimer disease (right). We show in Section \ref{['sec:experiments']} that we obtain a lower Fréchet inception distance (fid) compared to other interpolation schemes.
• Figure 3: Interpolation curves computed for different values of $\lambda$ in eq. \ref{['eq:geodesic_lagrange_simplify']} with a Euclidean background metric, where the data density is approximated by a Gaussian mixture model (gmm) and kernel density estimator (kde) shown in the background color, respectively, for synthetic data (black). The mean log-likelihoods of the estimated curves are denoted $\mathcal{L}$, where the first number is for the gmm, while the latter number is for the kde.
• Figure 4: Computed interpolations in noise space for ControlNet zhang2023addingconditionalcontroltexttoimage for two images of cat of size $768 \times 768 \times 3$ with an Euclidean background metric, where the regularizer function in eq. \ref{['eq:reg_fun']} is the density of the $\chi^{2}$-distribution on the norm of the grid points $z_{0:N}$. The left hand side shows the Euclidean energy of the computed curves using eq. \ref{['eq:energy']}. We see that larger values of $\lambda$ reduces smoothness, but increases the image quality as expected.
• Figure 5: Computed interpolations for ControlNet zhang2023addingconditionalcontroltexttoimage for eagle images of size $768 \times 768 \times 3$ similar to the experiment by zheng2024noisediffusioncorrectingnoiseimage. For ProbGEORCE we consider a Euclidean background metric, where the regularizer function in eq. \ref{['eq:reg_fun']} is the density of the $\chi^{2}$-distribution on the norm of the grid points $z_{0:N}$. The left hand side shows the the Euclidean energy of the computed curves using eq. \ref{['eq:energy']}. We see that ProbGEORCE obtains similar realism in images with smoother transition compared to NoiseDiffusion. Note that NoiseDiffusion assumes that the limiting distribution is isotropic Gaussian unlike our method.

Theorems & Definitions (14)

• Proposition 3.1
• proof
• Proposition 3.2: Update Scheme
• Corollary 3.2.1: Euclidean Update Scheme
• Proposition 3.3: Local Metric
• proof
• Proposition B.1
• proof
• Proposition B.2: Global Convergence
• proof