Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolution of Gaussians in the Hellinger-Kantorovich-Boltzmann gradient flow

Matthias Liero, Alexander Mielke, Oliver Tse, Jia-Jie Zhu

Abstract

This study leverages the basic insight that the gradient-flow equation associated with the relative Boltzmann entropy, in relation to a Gaussian reference measure within the Hellinger-Kantorovich (HK) geometry, preserves the class of Gaussian measures. This invariance serves as the foundation for constructing a reduced gradient structure on the parameter space characterizing Gaussian densities. We derive explicit ordinary differential equations that govern the evolution of mean, covariance, and mass under the HK-Boltzmann gradient flow. The reduced structure retains the additive form of the HK metric, facilitating a comprehensive analysis of the dynamics involved. We explore the geodesic convexity of the reduced system, revealing that global convexity is confined to the pure transport scenario, while a variant of sublevel semi-convexity is observed in the general case. Furthermore, we demonstrate exponential convergence to equilibrium through Polyak-Lojasiewicz-type inequalities, applicable both globally and on sublevel sets. By monitoring the evolution of covariance eigenvalues, we refine the decay rates associated with convergence. Additionally, we extend our analysis to non-Gaussian targets exhibiting strong log-lambda-concavity, corroborating our theoretical results with numerical experiments that encompass a Gaussian-target gradient flow and a Bayesian logistic regression application.

Evolution of Gaussians in the Hellinger-Kantorovich-Boltzmann gradient flow

Abstract

This study leverages the basic insight that the gradient-flow equation associated with the relative Boltzmann entropy, in relation to a Gaussian reference measure within the Hellinger-Kantorovich (HK) geometry, preserves the class of Gaussian measures. This invariance serves as the foundation for constructing a reduced gradient structure on the parameter space characterizing Gaussian densities. We derive explicit ordinary differential equations that govern the evolution of mean, covariance, and mass under the HK-Boltzmann gradient flow. The reduced structure retains the additive form of the HK metric, facilitating a comprehensive analysis of the dynamics involved. We explore the geodesic convexity of the reduced system, revealing that global convexity is confined to the pure transport scenario, while a variant of sublevel semi-convexity is observed in the general case. Furthermore, we demonstrate exponential convergence to equilibrium through Polyak-Lojasiewicz-type inequalities, applicable both globally and on sublevel sets. By monitoring the evolution of covariance eigenvalues, we refine the decay rates associated with convergence. Additionally, we extend our analysis to non-Gaussian targets exhibiting strong log-lambda-concavity, corroborating our theoretical results with numerical experiments that encompass a Gaussian-target gradient flow and a Bayesian logistic regression application.
Paper Structure (29 sections, 21 theorems, 174 equations, 4 figures, 1 algorithm)

This paper contains 29 sections, 21 theorems, 174 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. $\mathsf H\!\!\mathsf K_{\alpha,\beta}$-Boltzmann gradient system over Gaussian measures
  3. Overview of the paper
  4. ODE system for (Si,m,ka)
  5. Working with a simpler parametrization
  6. Returning to standard coordinates
  7. Reduced gradient system
  8. General reduction of gradient structures
  9. Reduced Onsager operators
  10. Reduced gradient system
  11. Reduction to normalized Gaussians
  12. Extension to non-Gaussian target measure $\uppi$
  13. Geodesic convexity of the reduced gradient system
  14. Geodesic convexity and the metric Hessian
  15. The Hessian for the Gaussian family
  16. ...and 14 more sections

Key Result

Proposition 2.1

The unique solution $t\to \rho(t)$ of equation eq:I.HK.PDE with initial condition $\rho(0,\cdot)= \Upphi(\mathsf q_0) \in \mathfrak G\subset {\mathcal{M}}({\mathbb R}^d)$ is given by $\rho(t,\cdot)= \Upphi(\mathsf q(t))$, where $t\mapsto \mathsf q(t) \in \mathfrak Q$ is the unique solution of eq:ODE

Figures (4)

  • Figure 5.1: Plot of the auxiliary function $H(\delta,\beta;y)$ in Lemma \ref{['lem:PL-h']}.
  • Figure 6.1: (Top) Evolution of the Gaussian probability measure $\mu_k$ (black contours) towards the target Gaussian measure $\uppi$ (blue contour). We simulated the evolution using Algorithm \ref{['alg:JKO']} that implements the $\mathsf H\!\!\mathsf K$-Gaussian gradient descent. (Bottom) Evolution of the mass variable $\kappa_k$ in both log and linear scale.
  • Figure 6.2: (Top) Evolution of the KL divergence ${\mathcal{H}}_{\mathrm B}(\mu_k|\uppi)$ (blue line) for the three different gradient flows of probability measures defined on ${\mathbb R}^{100}$: the Hellinger-Kantorovich-Gaussian gradient flow (blue; abbreviated as BWFR (Bures-Wasserstein-Fisher-Rao)), the Fisher-Rao gradient flow (green), and the Bures-Wasserstein gradient flow (orange). We observe that the Fisher-Rao gradient flow has the most rapid convergence in KL divergence asymptotically, while the Bures-Wasserstein gradient flow is initially faster. The HK-Gaussian gradient flow is between those two. (Bottom) Evolution of the mass variable $\kappa_k$ for the gradient flow of probability measures defined on ${\mathbb R}^{100}$.
  • Figure 6.3: (Left) Evolution of the Gaussian probability measure $\rho_k$ (black ellipsoidal contours) for the Bayesian logistic regression task. The background color represents the log-density of the Gaussian measure. The darker shade corresponds to higher density. The contours of the true target measure $\uppi$ is not ellipsoidal in the Bayesian logistic regression task. (Right) Samples of the classification decision boundary from the optimized Gaussian measure.

Theorems & Definitions (34)

  • Proposition 2.1: Gaussian solutions for \ref{['eq:I.HK.PDE']}
  • Proposition 2.2: Induced GFE on $\mathfrak P$
  • Remark 2.3: Explicit solutions
  • Remark 2.4: Long-time asymptotics
  • Theorem 3.1: Reduction of Riemannian GS
  • Theorem 3.2: Reduction of Onsager GS
  • Proposition 3.3: Formula for reduced Onsager operator
  • Remark 3.4: Reduction in terms of the Schur complement
  • Remark 3.5
  • Lemma 3.6
  • ...and 24 more