Inverse Problems Over Probability Measure Space

TL;DR

This work formulates and analyzes inverse problems on the space of probability measures, where the forward map $\mathcal{G}$ transports $\rho_x$ to $\rho_y$ via $\mathcal{G}_{\#}\rho_x$. By separating the analysis into overdetermined, underdetermined, and regularized regimes, it shows that the reconstruction depends critically on the chosen misfit $\mathcal{D}$, regularizer, and objective $\mathcal{E}$: $\phi$-divergences recover conditional distributions while Wasserstein distances yield marginal reconstructions in the overdetermined setting; entropy minimization and moment-based criteria produce piecewise constant or least-norm pullbacks in the underdetermined case, respectively. In the regularized framework, entropy-entropy pairs lead to piecewise-defined, prior-informed reconstructions, whereas $W_p$-moment pairs reproduce a least-norm-like solution under regularization. The paper also provides explicit linear and simple nonlinear examples to illustrate these contrasting behaviors and derives closed-form expressions in several settings, linking to classical inverse problems such as Tikhonov regularization in the Wasserstein space.

Abstract

Define a forward problem as $ρ_y = G_\#ρ_x$, where the probability distribution $ρ_x$ is mapped to another distribution $ρ_y$ using the forward operator $G$. In this work, we investigate the corresponding inverse problem: Given $ρ_y$, how to find $ρ_x$? Depending on whether $ G$ is overdetermined or underdetermined, the solution can have drastically different behavior. In the overdetermined case, we formulate a variational problem $\min_{ρ_x} D( G_\#ρ_x, ρ_y)$, and find that different choices of the metric $ D$ significantly affect the quality of the reconstruction. When $ D$ is set to be the Wasserstein distance, the reconstruction is the marginal distribution, while setting $ D$ to be a $φ$-divergence reconstructs the conditional distribution. In the underdetermined case, we formulate the constrained optimization $\min_{\{ G_\#ρ_x=ρ_y\}} E[ρ_x]$. The choice of $ E$ also significantly impacts the construction: setting $ E$ to be the entropy gives us the piecewise constant reconstruction, while setting $ E$ to be the second moment, we recover the classical least-norm solution. We also examine the formulation with regularization: $\min_{ρ_x} D( G_\#ρ_x, ρ_y) + α\mathsf R[ρ_x]$, and find that the entropy-entropy pair leads to a regularized solution that is defined in a piecewise manner, whereas the $W_2$-$W_2$ pair leads to a least-norm solution where $W_2$ is the 2-Wasserstein metric.

Figures (3)

• Figure 1: We denote $\rho_y^* := \mathcal{G}_\#\rho_x^*$; (a): the noisy data distribution with mass supported outside the range $\mathcal{R}$; (b): the reconstructed data distribution with $\mathcal{D}$ being the $\phi$-divergence; (c): the reconstructed data distribution with $\mathcal{D}$ being the $p$-Wasserstein metric.
• Figure 1: The data distribution $\mu_r= \mathcal{U}([0,1])$ is the uniform distribution in $[0, 1]$. (a): the optimizer to \ref{['eqn:least_norm']} when $\mathcal{E}$ is the entropy. The density is constant when restricted to level sets of $\mathcal{G}$; (b) the optimizer to \ref{['eqn:least_norm']} when $\mathcal{E}$ is the second-order moment. The parameter distribution is concentrated at the element of the minimum norm on each level set of $\mathcal{G}$.
• Figure 2: A comparison of the optimal distributions under different setups: (a) shows the result without regularization and the minimum norm cost $\mathcal{E}[\rho_x] = \int |x|^2 \mathrm{d} \rho_x$, while (b) shows the result for the regularized version with $(\mathcal{D},\mathsf{R}) = (\mathcal{W}_2^2, \int |x|^2 \mathrm{d} \rho_x(x))$ and $\alpha\neq 0$.

Theorems & Definitions (19)

• Definition 1
• Theorem 2
• Proof 1: Proof of Theorem \ref{['thm:phi_divergence']}
• Definition 3
• Theorem 4
• Proof 2
• Theorem 1
• Proof 3: Proof of Theorem \ref{['thm:KL-under']}
• Theorem 2
• Proof 4