Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A numerical algorithm with linear complexity for Multi-marginal Optimal Transport with $L^1$ Cost

Chunhui Chen, Jing Chen, Baojia Luo, Shi Jin, Hao Wu

TL;DR

The authors address the prohibitive cost of solving multi-marginal optimal transport with $L^1$ cost by developing a linear-time algorithm that replaces dense tensor-vector products with recursive, dynamic-programming based computations. By exploiting the kernel structure $K_{ijk}=e^{-c_{ijk}/oldsymbol{ε}}$ and partitioning index triples into six order-sets to remove absolute values, they derive two fast tensor-vector solvers (FTVP-1 and FTVP-2) and integrate them into a fast Sinkhorn framework (FS-3), with an optional log-domain stabilized variant (FTVP-LOG). The method extends naturally to higher dimensions and arbitrary numbers of marginals ($l$-marginal MMOT), preserving $O(N)$ complexity per update, and is validated through extensive 1D, 2D, and image-matching experiments showing substantial speedups while maintaining accuracy. Overall, this work enables scalable MMOT computations in practical settings such as image processing, tomography, and machine learning by reducing the dominant bottleneck from $O(N^l)$ to $O(N)$.

Abstract

Numerically solving multi-marginal optimal transport (MMOT) problems is computationally prohibitive, even for moderate-scale instances involving $l\ge4$ marginals with support sizes of $N\ge1000$. The cost in MMOT is represented as a tensor with $N^l$ elements. Even accessing each element once incurs a significant computational burden. In fact, many algorithms require direct computation of tensor-vector products, leading to a computational complexity of $O(N^l)$ or beyond. In this paper, inspired by our previous work [$Comm. \ Math. \ Sci.$, 20 (2022), pp. 2053 - 2057], we observe that the costly tensor-vector products in the Sinkhorn Algorithm can be computed with a recursive process by separating summations and dynamic programming. Based on this idea, we propose a fast tensor-vector product algorithm to solve the MMOT problem with $L^1$ cost, achieving a miraculous reduction in the computational cost of the entropy regularized solution to $O(N)$. Numerical experiment results confirm such high performance of this novel method which can be several orders of magnitude faster than the original Sinkhorn algorithm.

A numerical algorithm with linear complexity for Multi-marginal Optimal Transport with $L^1$ Cost

TL;DR

The authors address the prohibitive cost of solving multi-marginal optimal transport with cost by developing a linear-time algorithm that replaces dense tensor-vector products with recursive, dynamic-programming based computations. By exploiting the kernel structure and partitioning index triples into six order-sets to remove absolute values, they derive two fast tensor-vector solvers (FTVP-1 and FTVP-2) and integrate them into a fast Sinkhorn framework (FS-3), with an optional log-domain stabilized variant (FTVP-LOG). The method extends naturally to higher dimensions and arbitrary numbers of marginals (-marginal MMOT), preserving complexity per update, and is validated through extensive 1D, 2D, and image-matching experiments showing substantial speedups while maintaining accuracy. Overall, this work enables scalable MMOT computations in practical settings such as image processing, tomography, and machine learning by reducing the dominant bottleneck from to .

Abstract

Numerically solving multi-marginal optimal transport (MMOT) problems is computationally prohibitive, even for moderate-scale instances involving marginals with support sizes of . The cost in MMOT is represented as a tensor with elements. Even accessing each element once incurs a significant computational burden. In fact, many algorithms require direct computation of tensor-vector products, leading to a computational complexity of or beyond. In this paper, inspired by our previous work [, 20 (2022), pp. 2053 - 2057], we observe that the costly tensor-vector products in the Sinkhorn Algorithm can be computed with a recursive process by separating summations and dynamic programming. Based on this idea, we propose a fast tensor-vector product algorithm to solve the MMOT problem with cost, achieving a miraculous reduction in the computational cost of the entropy regularized solution to . Numerical experiment results confirm such high performance of this novel method which can be several orders of magnitude faster than the original Sinkhorn algorithm.
Paper Structure (15 sections, 72 equations, 5 figures, 6 tables, 6 algorithms)

This paper contains 15 sections, 72 equations, 5 figures, 6 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Multi-marginal optimal transport and the Sinkhorn algorithm
  3. An algorithm for MMOT with linear complexity
  4. Fast tensor-vector product of $\mathcal{K} \times_i \boldsymbol{\phi} \times_j \boldsymbol{\psi}$
  5. Fast tensor-vector product of $(\mathcal{C} \odot \mathcal{K}) \times_i \boldsymbol{\phi} \times_j \boldsymbol{\psi}$
  6. Algorithm implementation
  7. Various Generalization
  8. High dimension generalization
  9. $l$-marginal generalization
  10. Numerical Experiments
  11. 1D random distribution
  12. The Ricker wavelet
  13. 2D random distribution
  14. Multiple image matching problem
  15. Conclusion

Figures (5)

  • Figure 1: The 1D random distribution problem. (a): The computational time of our algorithm and the Sinkhorn algorithm with different numbers of grid points $N$. (b): The computational time required to reach the corresponding marginal error for $N = 80$ under $\varepsilon=0.1$ (purple) and $\varepsilon=0.05$ (green).
  • Figure 2: The Ricker wavelet problem. (a): The computational time required to reach the corresponding marginal error for $N = 80$ under $\varepsilon=0.1$ (purple) and $\varepsilon=0.05$ (green). (b): The comparison between our algorithms with and without the log-domain stabilization for $N = 100, \varepsilon= 0.001$.
  • Figure 3: The 2D random distribution problem. (a): The comparison between the Sinkhorn algorithm and our algorithm with different numbers of grid points $N$. (b): The computational time required to reach the corresponding marginal error for $N = 10$ under $\varepsilon=0.1$ (purple) and $\varepsilon=0.01$ (green).
  • Figure 4: Illustration of images from DOTmark dataset.
  • Figure 5: Multiple image matching problem. (a): The computational time required to reach the corresponding marginal error for $N = 16$ under $\varepsilon=0.1$ (purple) and $\varepsilon=0.05$ (green). (b): The comparison between our algorithms with and without the log-domain stabilization for $N = 32, \varepsilon= 0.0005$.

Theorems & Definitions (1)

  • remark 1