Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PINS: Proximal Iterations with Sparse Newton and Sinkhorn for Optimal Transport

Di Wu, Ling Liang, Haizhao Yang

TL;DR

The paper tackles the challenge of solving large-scale discrete optimal transport with high accuracy while avoiding numerical instability from entropic regularization. It introduces PINS, a two-phase proximal framework that first uses the Sinkhorn algorithm to obtain a well-conditioned start and then performs a sparse Newton refinement with Hessian sparsification, conjugate gradient solves, and line search. The authors establish global convergence and provide sparsification analysis showing bounded Hessian sparsity, with empirical results on synthetic and augmented MNIST datasets demonstrating near $10^{-10}$ accuracy and substantial speedups over Sinkhorn-type methods. The approach offers a scalable, robust pathway to exact-like OT solutions in large-scale settings, reducing sensitivity to the regularization parameter and enhancing practical impact in ML applications.

Abstract

Optimal transport (OT) is a critical problem in optimization and machine learning, where accuracy and efficiency are paramount. Although entropic regularization and the Sinkhorn algorithm improve scalability, they frequently encounter numerical instability and slow convergence, especially when the regularization parameter is small. In this work, we introduce Proximal Iterations with Sparse Newton and Sinkhorn methods (PINS) to efficiently compute highly accurate solutions for large-scale OT problems. A reduced computational complexity through overall sparsity and global convergence are guaranteed by rigorous theoretical analysis. Our approach offers three key advantages: it achieves accuracy comparable to exact solutions, progressively accelerates each iteration for greater efficiency, and enhances robustness by reducing sensitivity to regularization parameters. Extensive experiments confirm these advantages, demonstrating superior performance compared to related methods.

PINS: Proximal Iterations with Sparse Newton and Sinkhorn for Optimal Transport

TL;DR

The paper tackles the challenge of solving large-scale discrete optimal transport with high accuracy while avoiding numerical instability from entropic regularization. It introduces PINS, a two-phase proximal framework that first uses the Sinkhorn algorithm to obtain a well-conditioned start and then performs a sparse Newton refinement with Hessian sparsification, conjugate gradient solves, and line search. The authors establish global convergence and provide sparsification analysis showing bounded Hessian sparsity, with empirical results on synthetic and augmented MNIST datasets demonstrating near accuracy and substantial speedups over Sinkhorn-type methods. The approach offers a scalable, robust pathway to exact-like OT solutions in large-scale settings, reducing sensitivity to the regularization parameter and enhancing practical impact in ML applications.

Abstract

Optimal transport (OT) is a critical problem in optimization and machine learning, where accuracy and efficiency are paramount. Although entropic regularization and the Sinkhorn algorithm improve scalability, they frequently encounter numerical instability and slow convergence, especially when the regularization parameter is small. In this work, we introduce Proximal Iterations with Sparse Newton and Sinkhorn methods (PINS) to efficiently compute highly accurate solutions for large-scale OT problems. A reduced computational complexity through overall sparsity and global convergence are guaranteed by rigorous theoretical analysis. Our approach offers three key advantages: it achieves accuracy comparable to exact solutions, progressively accelerates each iteration for greater efficiency, and enhances robustness by reducing sensitivity to regularization parameters. Extensive experiments confirm these advantages, demonstrating superior performance compared to related methods.

Paper Structure

This paper contains 28 sections, 3 theorems, 29 equations, 5 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Computational Optimal Transport.
  4. Proximal Methods.
  5. Applications of OT.
  6. Methodology
  7. The Sinkhorn Algorithm
  8. Issues in Sinkhorn-Type Algorithms
  9. PINS
  10. Sparsification
  11. Conjugate Gradient Method
  12. Line Search
  13. Theoretical Guarantee
  14. Global Convergence
  15. Sparsification Analysis
  16. ...and 13 more sections

Key Result

Theorem 4.1

Let $X^k$ be the sequence generated by (eq:x_inner). Then $\{X^k\}$ converges to the optimal solution of the original OT problem (ot).

Figures (5)

  • Figure 1: Comparison of PINS and Newton accelerated methods without EPPA loops on synthetic datasets. The vertical axis represents the logarithmic error between the computed cost and the exact cost. The gray dashed line indicates the converging solution of the Newton accelerated methods for clarity.
  • Figure 2: Comparison of PINS and Newton accelerated methods without EPPA loops on (augmented) MNIST datasets.
  • Figure 3: The comparison between PINS and Sinkhorn with EPPA on synthetic dataset.
  • Figure 4: The comparison between PINS and Sinkhorn with EPPA on MNIST dataset.
  • Figure 5: The comparison among different regularization parameters of experiments on two datasets.

Theorems & Definitions (10)

  • Theorem 4.1
  • Remark 4.2
  • proof
  • Lemma 4.3
  • Theorem 4.4
  • Remark 4.5
  • Remark 4.6
  • proof
  • Remark 5.1
  • Remark 5.2