Continuous normalizing flows on manifolds
Luca Falorsi
TL;DR
This work closes a gap for probabilistic modeling on non-Euclidean spaces by developing continuous normalizing flows directly on arbitrary smooth manifolds. It introduces a manifold-centric pipeline that parameterizes vector fields via generating sets, uses measure-theoretic foundations for densities, and employs cotangent lifts to backpropagate through flows, all while providing an unbiased divergence estimator. The approach is demonstrated on a broad spectrum of spaces, including spheres, Lie groups, Stiefel manifolds, and symmetric positive definite matrices, achieving accurate density matching with high effective sampling sizes. The framework’s intrinsic formulation and algebraic structure enable principled, scalable density modeling on complex geometries, with potential impact across physics, robotics, and geometry-aware machine learning.
Abstract
Normalizing flows are a powerful technique for obtaining reparameterizable samples from complex multimodal distributions. Unfortunately, current approaches are only available for the most basic geometries and fall short when the underlying space has a nontrivial topology, limiting their applicability for most real-world data. Using fundamental ideas from differential geometry and geometric control theory, we describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifolds. We propose a general methodology for parameterizing vector fields on these spaces and demonstrate how gradient-based learning can be performed. Additionally, we provide a scalable unbiased estimator for the divergence in this generalized setting. Experiments on a diverse selection of spaces empirically showcase the defined framework's ability to obtain reparameterizable samples from complex distributions.