Monge-Kantorovich Fitting With Sobolev Budgets

TL;DR

This work studies Monge–Kantorovich fitting where an $m$-dimensional measure is approximated by an $n$-dimensional measure under a Sobolev budget, formulating the problem as minimizing ${\mathscr J}_p(f)$ with ${\mathscr C}(f)\le\ell$ and connecting it to unequal-dimensional OT via closest-point projection. A gradient-like barycenter field is developed to capture the first variation of ${\mathscr J}_p$, enabling local improvements and proving an almost-strict monotonicity of the optimal value $J(\ell)$. Existence, regularity, and continuity properties are established, along with discretization schemes and consistency results. The paper also explores discretized toy simulations and outlines connections to generative learning, proposing a two-step factorization of WGANs in which Sobolev-regularized maps first approximate the target measure’s support, followed by reparametrizations that yield practical gains in training stability and interpolation. Collectively, these contributions provide a theoretically-grounded framework for Sobolev-budgeted low-dimensional fitting, with implications for manifold learning and regularized generative modeling.

Abstract

Given $m < n$, we consider the problem of ``best'' approximating an $n\text{-d}$ probability measure $ρ$ via an $m\text{-d}$ measure $ν$ such that $\mathrm{supp}\ ν$ has bounded total ``complexity.'' When $ρ$ is concentrated near an $m\text{-d}$ set we may interpret this as a manifold learning problem with noisy data. However, we do not restrict our analysis to this case, as the more general formulation has broader applications. We quantify $ν$'s performance in approximating $ρ$ via the Monge-Kantorovich (also called Wasserstein) $p$-cost $\mathbb{W}_p^p(ρ, ν)$, and constrain the complexity by requiring $\mathrm{supp}\ ν$ to be coverable by an $f : \mathbb{R}^{m} \to \mathbb{R}^{n}$ whose $W^{k,q}$ Sobolev norm is bounded by $\ell \geq 0$. This allows us to reformulate the problem as minimizing a functional $\mathscr J_p(f)$ under the Sobolev ``budget'' $\ell$. This problem is closely related to (but distinct from) principal curves with length constraints when $m=1, k = 1$ and an unsupervised analogue of smoothing splines when $k > 1$. New challenges arise from the higher-order differentiability condition. We study the ``gradient'' of $\mathscr J_p$, which is given by a certain vector field that we call the barycenter field, and use it to prove a nontrivial (almost) strict monotonicity result. We also provide a natural discretization scheme and establish its consistency. We use this scheme as a toy model for a generative learning task, and by analogy, propose novel interpretations for the role regularization plays in improving training.

Figures (39)

• Figure 1: Closest points for $N=250$ uniformly-sampled $\omega \in \Omega$.
• Figure 2: Closest point projections for $N=250$ points sampled according to $\rho = {\mathcal{N}}({\mathbf{0}}; [\sigma_X^2\ 0;\ 0 \sigma_Y^2])$ (truncated to $\Omega$), where $\sigma_X^2 = 1$, $\sigma_Y^2=9/121$.
• Figure 3: Adding two kinds of spikes.
• Figure 4: Example for $\Omega = B_1(0; \mathbb{R}^2)$ and $(\pm 1,0) \in \mathop{\mathrm{supp}}\nolimits( \rho)$.
• Figure 5: Example $f_0$, $f_{.5}$ and $f_1$ for which $Y_0 \not\equiv Y_1$.

Theorems & Definitions (59)

• Remark 1.1
• Lemma 1.2
• proof
• Definition 2.1
• Remark 2.2
• Lemma 2.3
• Proposition 2.4: General Sobolev Inequality
• Corollary 2.5
• Definition 2.6: Objective Function
• Proposition 2.7