Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-Probability Polynomial-Time Complexity of Restarted PDHG for Linear Programming

Zikai Xiong

TL;DR

This work establishes, for restarted primal-dual hybrid gradient methods solving linear programs, high-probability polynomial-time bounds under a probabilistic model with sub-Gaussian (and Gaussian) data. It reveals a two-stage convergence: an initial basis identification stage with bound roughly proportional to a dimension-dependent factor, followed by a fast local convergence stage with a log(1/ε) dependence. The analysis leverages non-asymptotic random-matrix theory to bound key quantities like the smallest singular value, condition numbers, and disparity measures of the optimal solution, and it is complemented by experimental validation of tail behavior and polynomial scaling. The results explain the practical efficiency of rPDHG and offer guidance for generating challenging LP instances via the disparity among optimal components. Overall, this work bridges theory and practice for a scalable first-order LP method, with implications for solver design and benchmark construction.

Abstract

The restarted primal-dual hybrid gradient method (rPDHG) is a first-order method that has recently received significant attention for its computational effectiveness in solving linear program (LP) problems. Despite its impressive practical performance, the theoretical iteration bounds for rPDHG can be exponentially poor. To shrink this gap between theory and practice, we show that rPDHG achieves polynomial-time complexity in a high-probability sense, under assumptions on the probability distribution from which the data instance is generated. We consider not only Gaussian distribution models but also sub-Gaussian distribution models as well. For standard-form LP instances with $m$ linear constraints and $n$ decision variables, we prove that rPDHG iterates settle on the optimal basis in $\widetilde{O}\left(\tfrac{n^{2.5}m^{0.5}}δ\right)$ iterations, followed by $O\left(\frac{n^{0.5}m^{0.5}}δ\ln\big(\tfrac{1}{\varepsilon}\big)\right)$ iterations to compute an $\varepsilon$-optimal solution. These bounds hold with probability at least $1-δ$ for $δ$ that is not exponentially small. The first-stage bound further improves to $\widetilde{O}\left(\frac{n^{2.5}}δ\right)$ in the Gaussian distribution model. Experimental results confirm the tail behavior and the polynomial-time dependence on problem dimensions of the iteration counts. As an application of our probabilistic analysis, we explore how the disparity among the components of the optimal solution bears on the performance of rPDHG, and we provide guidelines for generating challenging LP test instance.

High-Probability Polynomial-Time Complexity of Restarted PDHG for Linear Programming

TL;DR

This work establishes, for restarted primal-dual hybrid gradient methods solving linear programs, high-probability polynomial-time bounds under a probabilistic model with sub-Gaussian (and Gaussian) data. It reveals a two-stage convergence: an initial basis identification stage with bound roughly proportional to a dimension-dependent factor, followed by a fast local convergence stage with a log(1/ε) dependence. The analysis leverages non-asymptotic random-matrix theory to bound key quantities like the smallest singular value, condition numbers, and disparity measures of the optimal solution, and it is complemented by experimental validation of tail behavior and polynomial scaling. The results explain the practical efficiency of rPDHG and offer guidance for generating challenging LP instances via the disparity among optimal components. Overall, this work bridges theory and practice for a scalable first-order LP method, with implications for solver design and benchmark construction.

Abstract

The restarted primal-dual hybrid gradient method (rPDHG) is a first-order method that has recently received significant attention for its computational effectiveness in solving linear program (LP) problems. Despite its impressive practical performance, the theoretical iteration bounds for rPDHG can be exponentially poor. To shrink this gap between theory and practice, we show that rPDHG achieves polynomial-time complexity in a high-probability sense, under assumptions on the probability distribution from which the data instance is generated. We consider not only Gaussian distribution models but also sub-Gaussian distribution models as well. For standard-form LP instances with linear constraints and decision variables, we prove that rPDHG iterates settle on the optimal basis in iterations, followed by iterations to compute an -optimal solution. These bounds hold with probability at least for that is not exponentially small. The first-stage bound further improves to in the Gaussian distribution model. Experimental results confirm the tail behavior and the polynomial-time dependence on problem dimensions of the iteration counts. As an application of our probabilistic analysis, we explore how the disparity among the components of the optimal solution bears on the performance of rPDHG, and we provide guidelines for generating challenging LP test instance.
Paper Structure (31 sections, 31 theorems, 94 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 31 sections, 31 theorems, 94 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Outline and contributions
  3. Related works
  4. Notation
  5. Restarted PDHG for Linear Programming
  6. Restarted Primal-dual hybrid gradient method (rPDHG)
  7. High-Probability Computational Guarantees of rPDHG
  8. Probabilistic linear programming model
  9. High-probability performance guarantees for LP instances with sub-Gaussian input data
  10. High-probability performance guarantees for LP instances with Gaussian input data
  11. Proof of Theorem \ref{['thm:complexity-random']}
  12. Lemmas of random variables and random matrices
  13. Worst-case iteration bounds of rPDHG
  14. Proof of Theorem \ref{['thm:complexity-random']}
  15. Proof of Theorem \ref{['thm:complexity-gaussian']}
  16. ...and 16 more sections

Key Result

Lemma 3.1

For an instance of the probabilistic model, ${\mathcal{X}}^\star$, ${\mathcal{Y}}^\star$ and ${\mathcal{S}}^\star$ are all singletons with ${\mathcal{X}}^\star=\{\hat{x}\}$ and ${\mathcal{S}}^\star = \{\hat{s}\}$ if and only if $B$ is full-rank and $(\hat{x},\hat{s})$ satisfies the strict complement

Figures (3)

  • Figure 1: Number of iterations required to complete Stages I and II of at least a $(1-\delta)$ fraction of instances
  • Figure 2: The median values and interquartile range (IQR) of Stage-I and Stage-II iteration counts for various dimensions $n$.
  • Figure 3: The median values and the interquartile range (IQR) of Stage-I and Stage-II iteration counts for the random LP instances with various values of $\phi$ and $\min_{1\le i \le n} (x^\star_i + s^\star_i)$.

Theorems & Definitions (63)

  • Definition 3.1: Sub-Gaussian random variable
  • Example 3.1: Gaussian random variable
  • Example 3.2: Bounded random varibale
  • Definition 3.2: Sub-Gaussian matrix
  • Definition 3.3: Nonnegative absolutely continuous sub-Gaussian vector
  • Definition 3.4: Probabilistic model
  • Lemma 3.1
  • Lemma 3.2
  • Definition 3.5: $\varepsilon$-optimal solutions
  • Theorem 3.1
  • ...and 53 more