Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Batched First-Order Methods for Parallel LP Solving in MIP

Nicolas Blin, Stefano Gualandi, Christopher Maes, Andrea Lodi, Bartolomeo Stellato

TL;DR

We address the challenge of solving many related LPs arising in MIP by introducing a batched first-order method based on the primal-dual hybrid gradient (PDHG) algorithm, extended with Halpern restarts and per-subproblem batching to run efficiently on GPUs. The approach reformulates LPs into a saddle-point problem and exploits matrix-matrix operations to maximize GPU throughput, enabling exact or near-exact solutions for multiple subproblems such as full strong branching and OBBT. We demonstrate substantial speedups over traditional simplex-based methods, integrate BatchLP into cuOpt, and show promising results on MIPLIB 2017 instances, highlighting both the practical potential and the limitations dependent on problem structure and hardware. Overall, this work provides a scalable, GPU-optimized pathway for incorporating batched LP solves into MIP workflows, potentially reshaping how subproblem computations are allocated across CPU and GPU resources.

Abstract

We present a batched first-order method for solving multiple linear programs in parallel on GPUs. Our approach extends the primal-dual hybrid gradient algorithm to efficiently solve batches of related linear programming problems that arise in mixed-integer programming techniques such as strong branching and bound tightening. By leveraging matrix-matrix operations instead of repeated matrix-vector operations, we obtain significant computational advantages on GPU architectures. We demonstrate the effectiveness of our approach on various case studies and identify the problem sizes where first-order methods outperform traditional simplex-based solvers depending on the computational environment one can use. This is a significant step for the design and development of integer programming algorithms tightly exploiting GPU capabilities where we argue that some specific operations should be allocated to GPUs and performed in full instead of using light-weight heuristic approaches on CPUs.

Batched First-Order Methods for Parallel LP Solving in MIP

TL;DR

We address the challenge of solving many related LPs arising in MIP by introducing a batched first-order method based on the primal-dual hybrid gradient (PDHG) algorithm, extended with Halpern restarts and per-subproblem batching to run efficiently on GPUs. The approach reformulates LPs into a saddle-point problem and exploits matrix-matrix operations to maximize GPU throughput, enabling exact or near-exact solutions for multiple subproblems such as full strong branching and OBBT. We demonstrate substantial speedups over traditional simplex-based methods, integrate BatchLP into cuOpt, and show promising results on MIPLIB 2017 instances, highlighting both the practical potential and the limitations dependent on problem structure and hardware. Overall, this work provides a scalable, GPU-optimized pathway for incorporating batched LP solves into MIP workflows, potentially reshaping how subproblem computations are allocated across CPU and GPU resources.

Abstract

We present a batched first-order method for solving multiple linear programs in parallel on GPUs. Our approach extends the primal-dual hybrid gradient algorithm to efficiently solve batches of related linear programming problems that arise in mixed-integer programming techniques such as strong branching and bound tightening. By leveraging matrix-matrix operations instead of repeated matrix-vector operations, we obtain significant computational advantages on GPU architectures. We demonstrate the effectiveness of our approach on various case studies and identify the problem sizes where first-order methods outperform traditional simplex-based solvers depending on the computational environment one can use. This is a significant step for the design and development of integer programming algorithms tightly exploiting GPU capabilities where we argue that some specific operations should be allocated to GPUs and performed in full instead of using light-weight heuristic approaches on CPUs.
Paper Structure (25 sections, 12 equations, 4 figures, 3 tables)

This paper contains 25 sections, 12 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature review
  3. Problem formulation
  4. Primal-dual hybrid gradient to solve a single instance
  5. Restarts.
  6. Step-size and primal weight.
  7. Stopping criteria.
  8. Infeasibility detection.
  9. Solving batches of problems
  10. Restarts.
  11. Stopping criteria and infeasibility detection.
  12. Matrix-matrix products and optimal batch size.
  13. Naive batched implementation.
  14. Full strong branching
  15. Optimization-based bound tightening
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Sparse Matrix-Matrix Multiplies vs Batch Size on csched007. Total time for 10 $AX$ and 10 $A^T Y$ operations (blue circle markers on the left y-axis) and time/column (red square markers on the right y-axis).
  • Figure 2: Speedup of FSB runtime vs. 8-thread CPU baseline. GPU achieves 12--489$\times$ speedup. Full results in Table \ref{['tab:cuopt:threads']}.
  • Figure 3: OBBT speedup of BatchLP vs. Gurobi dual simplex on neural network verification instances. Average speedup: 25.7$\times$. Full results in Table \ref{['tab:obbt']}.
  • Figure 4: MIPLIB 2017 speedup of BatchLP vs. cuOpt dual simplex (28 threads). Values below 1 indicate dual simplex is faster. Full results in Table \ref{['tab:miplib']}.