Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Linear Programs with Fast Online Learning Algorithms

Wenzhi Gao, Dongdong Ge, Chunlin Sun, Yinyu Ye

TL;DR

This paper presents fast first-order methods for solving linear programs (LPs) approximately and introduces a variable-duplication technique that reduces the optimality gap and constraint violation by a factor of $\sqrt{K}$.

Abstract

This paper presents fast first-order methods for solving linear programs (LPs) approximately. We adapt online linear programming algorithms to offline LPs and obtain algorithms that avoid any matrix multiplication. We also introduce a variable-duplication technique that copies each variable $K$ times and reduces the optimality gap and constraint violation by a factor of $\sqrt{K}$. Furthermore, we show how online algorithms can be effectively integrated into sifting, a column generation scheme for large-scale LPs. Numerical experiments demonstrate that our methods can serve as either an approximate direct solver, or an initialization subroutine for exact LP solving.

Solving Linear Programs with Fast Online Learning Algorithms

TL;DR

This paper presents fast first-order methods for solving linear programs (LPs) approximately and introduces a variable-duplication technique that reduces the optimality gap and constraint violation by a factor of .

Abstract

This paper presents fast first-order methods for solving linear programs (LPs) approximately. We adapt online linear programming algorithms to offline LPs and obtain algorithms that avoid any matrix multiplication. We also introduce a variable-duplication technique that copies each variable times and reduces the optimality gap and constraint violation by a factor of . Furthermore, we show how online algorithms can be effectively integrated into sifting, a column generation scheme for large-scale LPs. Numerical experiments demonstrate that our methods can serve as either an approximate direct solver, or an initialization subroutine for exact LP solving.

Paper Structure

This paper contains 63 sections, 9 theorems, 92 equations, 4 figures, 7 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. First-order Methods for Linear Programming
  3. Simple and Fast Online LP
  4. Contributions
  5. Other Related Work
  6. Outline of the Paper
  7. Problem Setup and Assumptions
  8. Notations
  9. Linear Programming and Duality
  10. Simple and Fast Online Algorithms
  11. Assumptions
  12. Performance Measure
  13. Fast Online Algorithms for Offline LPs
  14. Online Explicit Update
  15. Online Implicit Update
  16. ...and 48 more sections

Key Result

Lemma 1

Under assumptions A1 to A3, if we take $\gamma_k \equiv \gamma$, then solution $\hat{\mathbf{x}}$ output by Algorithm alg:online using explicit subgradient update eqn:explicit satisfies where $\mathbb{E}[\cdot]$ is taken over the random permutation.

Figures (4)

  • Figure 1: First row from left to right $(m, n, K) \in \{(5, 10^2, 1), (5, 10^2, 8), (8, 10^3, 1), (8, 10^3, 8)\}$. Second row from left to right $(m, n, K) \in \{(16, 2\times10^3, 1), (16, 2\times10^3, 8), (32, 4\times10^3, 1), (32, 4\times10^3, 8)\}$. The x-axis represents $\tau$ parameter ranging from $10^{-2}$ to $1$; The y-axis represents the relative optimality.
  • Figure 2: From left to right $(m, n) \in \{(5, 10^2), (8, 10^3), (16, 2\times10^3), (32, 4\times10^3)\}$. The x-axis represents $K$ parameter ranging in $\{1, 2, 4, 8, 16, 32\}$. The y-axis represents the relative optimality.
  • Figure 3: From left to right: $(m, n) \in \{ (5, 100), (8, 1000), (16, 2000), (32, 4000) \}$ The axis represents $K$ parameter ranging from 1 to 128. y-axis, relative optimality gap
  • Figure 4: Convergence of the dual solution $\mathbf{y}^k$ to the optimal $\mathbf{y}^*$

Theorems & Definitions (23)

  • Remark 1
  • Remark 2
  • Lemma 1
  • Remark 3
  • Theorem 1
  • Remark 4
  • Remark 5
  • Remark 6
  • Lemma 2
  • Theorem 2
  • ...and 13 more