Optimal Transport with Tempered Exponential Measures

TL;DR

This paper shows that an extension of the latter to tempered exponential measures, a generalization of exponential families with indirect measure normalization, gets to a very convenient middle ground, with both very fast approximation algorithms and sparsity, which fits naturally in the unbalanced optimal transport problem setting.

Abstract

In the field of optimal transport, two prominent subfields face each other: (i) unregularized optimal transport, "à-la-Kantorovich", which leads to extremely sparse plans but with algorithms that scale poorly, and (ii) entropic-regularized optimal transport, "à-la-Sinkhorn-Cuturi", which gets near-linear approximation algorithms but leads to maximally un-sparse plans. In this paper, we show that an extension of the latter to tempered exponential measures, a generalization of exponential families with indirect measure normalization, gets to a very convenient middle ground, with both very fast approximation algorithms and sparsity, which is under control up to sparsity patterns. In addition, our formulation fits naturally in the unbalanced optimal transport problem setting.

Figures (10)

• Figure 1: In classical optimal transport (OT, left), regularized or not, marginals and the OT plan sought are in the probability simplex; the optimal solution solely depends on the metric properties of the supports. Entropic regularization balances the metric cost with an entropic cost, and the optimal solution has remarkable properties related to exponential families. In this paper (right), we lift the whole setting to families of measures generalizing exponential families: tempered exponential measures (TEMs). Specific properties that appear include unbalancedness and sparsity of the optimal solution (see text).
• Figure 2: Illustration of Theorem \ref{['th:sparseotem']}. Negative costs of matrix $\mathbf{M}'$ in the regularized measured cost \ref{['costregm']} are in blue, and those positive are in red. The sparsity results of the theorem are shown, where no arrow means no transport and a dashed arrow means a transport necessarily "small" (see text).
• Figure 3: The expected $t$-Sinkhorn distance relative to the Sinkhorn distance for different values of $t$. As $t\rightarrow 1$, the approximation error due to solving the unconstrained problem via alternating Bregman projections becomes smaller and the expected $t$-Sinkhorn distance converges to the Sinkhorn distance when $\lambda \rightarrow \infty$.
• Figure 4: Number of iterations to converge for (a) expected and (b) measured cost OT for different values of $\lambda$. Relative (to the OT solution) expected cost for the two cases, respectively, shown in (c) and (d).
• Figure 5: Transport plans induced by OT and the expected cost formulation for different values of $t$. The non-zero values are marked by a square. The EOT ($t=1$) induces a fully-dense plan. The sparsity of the solution increases by increasing $1 < t < 2$.

Theorems & Definitions (34)

• Remark 1
• Remark 2
• Definition 1
• Definition 2
• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Theorem 1
• Theorem 2