Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Wasserstein Gradient Flows of MMD Functionals with Distance Kernels under Sobolev Regularization

Richard Duong, Nicolaj Rux, Viktor Stein, Gabriele Steidl

TL;DR

This work studies Wasserstein gradient flows of $ ext{MMD}_K^2(\\mu,\\nu)$ with distance kernels $K(x,y)=\pm|x-y|$ in one dimension, leveraging the isometric quantile-embedding of $\\mathcal{P}_2(\\mathbb{R})$ into $L_2(0,1)$. The key challenge is the non-convexity of the positive-kernel functional; the authors introduce a Sobolev regularization $F_H(u)=\frac{\lambda}{2}\int_0^1 |u'(s)|^2 ds$ with Neumann BC and cone constraint to obtain well-posed gradient flows, yielding $ ilde F_{\\nu}^{\pm}$ that are amenable to minimizing-movement analysis. They prove existence results: a strong solution for the negative kernel (Theorem main_mmd) and a generalized minimizing movement for the positive kernel (Theorem main_mmd2), and provide explicit Dirac-to-Dirac and Uniform-to-Uniform examples to illustrate the dynamics and the rectification of a previously observed “dissipation-of-mass” defect. Numerically, an implicit Euler scheme coupled with a Neumann problem solver demonstrates that the regularized flows move mass toward the target and avoid unphysical spreading, offering a robust 1D framework with potential extensions to higher dimensions. Overall, the paper advances stable analysis and computation of MMD-driven Wasserstein gradient flows for non-smooth kernels by harnessing Sobolev regularization and quantile-function representations.

Abstract

We consider Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals $\text{MMD}_K^2(\cdot, ν)$ for positive and negative distance kernels $K(x,y) := \pm |x-y|$ and given target measures $ν$ on $\mathbb{R}$. Since in one dimension the Wasserstein space can be isometrically embedded into the cone $\mathcal C(0,1) \subset L_2(0,1)$ of quantile functions, Wasserstein gradient flows can be characterized by the solution of an associated Cauchy problem on $L_2(0,1)$. While for the negative kernel, the MMD functional is geodesically convex, this is not the case for the positive kernel, which needs to be handled to ensure the existence of the flow. We propose to add a regularizing Sobolev term $|\cdot|^2_{H^1(0,1)}$ corresponding to the Laplacian with Neumann boundary conditions to the Cauchy problem of quantile functions. Indeed, this ensures the existence of a generalized minimizing movement for the positive kernel. Furthermore, for the negative kernel, we demonstrate by numerical examples how the Laplacian rectifies a "dissipation-of-mass" defect of the MMD gradient flow.

Wasserstein Gradient Flows of MMD Functionals with Distance Kernels under Sobolev Regularization

TL;DR

This work studies Wasserstein gradient flows of with distance kernels in one dimension, leveraging the isometric quantile-embedding of into . The key challenge is the non-convexity of the positive-kernel functional; the authors introduce a Sobolev regularization with Neumann BC and cone constraint to obtain well-posed gradient flows, yielding that are amenable to minimizing-movement analysis. They prove existence results: a strong solution for the negative kernel (Theorem main_mmd) and a generalized minimizing movement for the positive kernel (Theorem main_mmd2), and provide explicit Dirac-to-Dirac and Uniform-to-Uniform examples to illustrate the dynamics and the rectification of a previously observed “dissipation-of-mass” defect. Numerically, an implicit Euler scheme coupled with a Neumann problem solver demonstrates that the regularized flows move mass toward the target and avoid unphysical spreading, offering a robust 1D framework with potential extensions to higher dimensions. Overall, the paper advances stable analysis and computation of MMD-driven Wasserstein gradient flows for non-smooth kernels by harnessing Sobolev regularization and quantile-function representations.

Abstract

We consider Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals for positive and negative distance kernels and given target measures on . Since in one dimension the Wasserstein space can be isometrically embedded into the cone of quantile functions, Wasserstein gradient flows can be characterized by the solution of an associated Cauchy problem on . While for the negative kernel, the MMD functional is geodesically convex, this is not the case for the positive kernel, which needs to be handled to ensure the existence of the flow. We propose to add a regularizing Sobolev term corresponding to the Laplacian with Neumann boundary conditions to the Cauchy problem of quantile functions. Indeed, this ensures the existence of a generalized minimizing movement for the positive kernel. Furthermore, for the negative kernel, we demonstrate by numerical examples how the Laplacian rectifies a "dissipation-of-mass" defect of the MMD gradient flow.

Paper Structure

This paper contains 22 sections, 10 theorems, 89 equations, 13 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Outline of the paper.
  3. Wasserstein Gradient Flows
  4. Flows on P2(Rd)
  5. Flows on P2(R)
  6. Distance Kernel MMDs with Sobolev Regularization
  7. General modeling for both kernels
  8. Flows of the negative kernel
  9. Flows of the positive kernel
  10. Numerics
  11. The Implicit Euler Scheme
  12. Technical implementation of solving the Neumann problem \ref{['eq:neumann-problem']}:
  13. Numerical Examples
  14. Dirac-to-Dirac:
  15. Technical implementation in the Dirac case:
  16. ...and 7 more sections

Key Result

Theorem 2.1

Let $\mathop{\mathrm{\mathcal{F}}}\nolimits \colon \mathop{\mathrm{\mathcal{P}}}\nolimits_2(\mathop{\mathrm{\mathbb R}}\nolimits) \to (- \infty, \infty]$ be bounded from below, lsc and $\lambda$-convex along geodesics for some $\lambda \in \mathop{\mathrm{\mathbb R}}\nolimits$. Let $\mu_0 \in \matho

Figures (13)

  • Figure 1: Top: MMD flow with negative distance kernel from $\delta_{-1}$ to $\delta_0$ has the "dissipation-of-mass" defect that the mass in $(-1,0)$ slowly dissipates towards zero, but does not vanish on any part of the interval. Bottom: Correction of the "dissipation-of-mass" defect by adding a Sobolev regularization term.
  • Figure 2: Functions $R_{\mu}^{+}$ (left), $R_{\mu}^{-}$ (middle) and $Q_{\mu}$ (right) for a discrete measure $\mu \coloneqq \sum_{k = 1}^{4} w_k \delta_{x_k}$ with $W_k \coloneqq \sum_{j = 1}^{k} w_j$. Source: see DSBHS2024.
  • Figure 3: Quantiles of the regularized and unregularized ${\mathop{\mathrm{\mathcal{F}}}\nolimits}_\nu^-$-flow from $\delta_{-1}$ ($Q_0 \equiv -1$) to $\delta_{0}$ ($Q_\nu \equiv 0$) for $\tau = 10^{-3}, \,\lambda = 10^{-2}$. The results strongly suggest an emergence of a Dirac at $0$ in the regularized flow.
  • Figure 4: Unregularized ${\mathcal{F}}_\nu^-$-flow from $\delta_{-1}$ to $\delta_{0}$ for $\tau = 10^{-3}$. The light color represents the absolute continuous part, while the dark color displays the singular part of the flow. The mass in $(-1,0)$ slowly dissipates, but never totally vanishes.
  • Figure 5: Regularized ${\mathcal{F}}_\nu^-$-flow from $\delta_{-1}$ to $\delta_{0}$ for $\tau = 10^{-3}, \,\lambda = 10^{-2}$. The "horns" at the boundary of the support are of height $Q_\nu'(0)^{-1} = Q_\nu'(1)^{-1} = \infty$. The support shifts towards the target.
  • ...and 8 more figures

Theorems & Definitions (24)

  • Theorem 2.1: Existence and uniqueness of Wasserstein gradient flows
  • Remark 2.2: Properties of $R_{\mu}^{\pm}$ and $Q_{\mu}$
  • Theorem 2.3
  • Theorem 2.4
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • Remark 3.3: $\widetilde{F}_{\nu}^\pm$ as associated functions of $\widetilde{\mathcal{F}}_{\nu}^\pm$
  • Theorem 3.4
  • proof
  • ...and 14 more