Principled Interpolation in Normalizing Flows

TL;DR

The paper addresses the problem that linear interpolation with Gaussian bases in normalizing flows often departs from the data manifold. It introduces a principled interpolation framework by restricting base distributions to fixed-norm manifolds on the simplex ($p=1$ with Dirichlet) or the unit hypersphere ($p=2$ with von Mises–Fisher), deriving efficient bijections and Jacobian computations, and employing appropriate interpolations (linear on the simplex and slerp on the sphere). Empirically, the von Mises–Fisher base improves bits-per-dimension on test data and yields more natural interpolations, with competitive FID/KID scores, while the Dirichlet base encounters numerical instability. Overall, the approach demonstrates that changing the base distribution to enforce a manifold yields better interpolation paths without sacrificing generative performance, offering a practical path to more faithful interpolations in flow-based models.

Abstract

Generative models based on normalizing flows are very successful in modeling complex data distributions using simpler ones. However, straightforward linear interpolations show unexpected side effects, as interpolation paths lie outside the area where samples are observed. This is caused by the standard choice of Gaussian base distributions and can be seen in the norms of the interpolated samples as they are outside the data manifold. This observation suggests that changing the way of interpolating should generally result in better interpolations, but it is not clear how to do that in an unambiguous way. In this paper, we solve this issue by enforcing a specific manifold and, hence, change the base distribution, to allow for a principled way of interpolation. Specifically, we use the Dirichlet and von Mises-Fisher base distributions on the probability simplex and the hypersphere, respectively. Our experimental results show superior performance in terms of bits per dimension, Fréchet Inception Distance (FID), and Kernel Inception Distance (KID) scores for interpolation, while maintaining the generative performance.

Figures (11)

• Figure 1: Illustration of different interpolation paths of points from a high-dimensional Gaussian. The figure also shows that, in high dimensions, points are not concentrated at the origin.
• Figure 2: Interpolation of samples from CelebA. Top: a linear interpolation path. The central images resemble features of the mean face as annotated in red. Bottom: an alternative interpolation path using a norm-correction. Note that the first and last three images are almost identical as annotated in blue. Right: decoded expectation of base distribution, i.e., the mean face.
• Figure 3: Two examples showing the issues caused by a norm-corrected linear interpolation (nclerp).
• Figure 4: An example of a one-dimensional normalizing flow.
• Figure 5: A stereographic projection mapping $\mathbf{z} \in \mathbb{R}^1$ to $\mathbf{s} \in \mathbb{S}^1_2$ using the north pole depicted as a black dot. The mean direction $\bm{\mu} \in \mathbb{S}^1_2$ is shown in orange.