Inexact Bregman Sparse Newton Method for Efficient Optimal Transport

TL;DR

The Inexact Bregman Sparse Newton method, which efficiently solves the exact OT problems through a sequence of semi-dual subproblems, and provides rigorous theoretical guarantees for the global convergence of the algorithm.

Abstract

Computing exact Optimal Transport (OT) distances for large-scale datasets is computationally prohibitive. While entropy-regularized alternatives offer speed, they sacrifice precision and frequently suffer from numerical instability in high-accuracy regimes. To address these limitations, we propose the Inexact Bregman Sparse Newton (IBSN) method, which efficiently solves the exact OT problems. Our approach utilizes a Bregman proximal point framework through a sequence of semi-dual subproblems. By solving these subproblems inexactly, we significantly reduce per-iteration complexity while maintaining a theoretical guarantee of convergence to the true optimal plan. To further accelerate the algorithm, we develop a sparse Newton-type solver for the subproblem and employ a Hessian sparsification strategy that drastically lowers memory and time costs without sacrificing accuracy. We provide rigorous theoretical guarantees for the global convergence of the algorithm. Extensive experiments demonstrate that IBSN consistently outperforms state-of-the-art methods in both computational speed and solution precision.

Figures (7)

• Figure 1: Performance of different algorithms on synthetic data. Top: Uniform setting. Bottom: Square setting.
• Figure 2: Performance of different algorithms on MNIST dataset (Top) and Fashion-MNIST dataset (Bottom).
• Figure 3: Performance of different algorithms on DOTmark dataset. Top: Image size $32 \times 32$. Bottom: Image size $64 \times 64$.
• Figure 4: Impact of the regularization parameter $\eta$ on IBSN, IBSink and PINS on synthetic data. Top: Uniform setting. Bottom: Square setting.
• Figure 5: Performance of different algorithms on EOT problem \ref{['opt: EOT']}. Top: Square setting. Middle: DOTmark with image size $32 \times 32$. Bottom: DOTmark with image size $64 \times 64$.

Theorems & Definitions (25)

• Lemma 3.1
• Theorem 3.2
• Theorem 3.3
• Theorem 3.4
• Lemma 4.1
• Theorem 4.2
• Theorem 4.3
• Proposition 4.4
• proof
• proof