Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IPM-LSTM: A Learning-Based Interior Point Method for Solving Nonlinear Programs

Xi Gao, Jinxin Xiong, Akang Wang, Qihong Duan, Jiang Xue, Qingjiang Shi

Abstract

Solving constrained nonlinear programs (NLPs) is of great importance in various domains such as power systems, robotics, and wireless communication networks. One widely used approach for addressing NLPs is the interior point method (IPM). The most computationally expensive procedure in IPMs is to solve systems of linear equations via matrix factorization. Recently, machine learning techniques have been adopted to expedite classic optimization algorithms. In this work, we propose using Long Short-Term Memory (LSTM) neural networks to approximate the solution of linear systems and integrate this approximating step into an IPM. The resulting approximate NLP solution is then utilized to warm-start an interior point solver. Experiments on various types of NLPs, including Quadratic Programs and Quadratically Constrained Quadratic Programs, show that our approach can significantly accelerate NLP solving, reducing iterations by up to 60% and solution time by up to 70% compared to the default solver.

IPM-LSTM: A Learning-Based Interior Point Method for Solving Nonlinear Programs

Abstract

Solving constrained nonlinear programs (NLPs) is of great importance in various domains such as power systems, robotics, and wireless communication networks. One widely used approach for addressing NLPs is the interior point method (IPM). The most computationally expensive procedure in IPMs is to solve systems of linear equations via matrix factorization. Recently, machine learning techniques have been adopted to expedite classic optimization algorithms. In this work, we propose using Long Short-Term Memory (LSTM) neural networks to approximate the solution of linear systems and integrate this approximating step into an IPM. The resulting approximate NLP solution is then utilized to warm-start an interior point solver. Experiments on various types of NLPs, including Quadratic Programs and Quadratically Constrained Quadratic Programs, show that our approach can significantly accelerate NLP solving, reducing iterations by up to 60% and solution time by up to 70% compared to the default solver.

Paper Structure

This paper contains 24 sections, 1 theorem, 35 equations, 6 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Approach
  4. The Classic IPM
  5. Approximating Solutions to Linear Systems
  6. The IPM-LSTM Approach
  7. Two-Stage Framework
  8. Experiments
  9. Experimental Settings
  10. Computational Results
  11. Performance Analysis of IPM-LSTM
  12. Limitations and Conclusions
  13. Implementation Details
  14. Proof of Proposition \ref{['prop1']}
  15. Datasets and Parameter setting
  16. ...and 9 more sections

Key Result

Proposition 1

If $(x^k,\lambda^k,z^k)$ is generated such that Assumption assump:bound is satisfied, let $(x^*,\lambda^{*}, z^{*})$ denote a limit point of the sequence $\{(x^k,\lambda^k,z^k)\}$, then $\{(x^k,\lambda^k,z^k)\}$ converges to $(x^*,\lambda^{*}, z^{*})$ and $F_0(x^*,\lambda^{*}, z^{*})=0$.

Figures (6)

  • Figure 1: An illustration of the IPM-LSTM approach.
  • Figure 2: The LSTM architecture for solving $\underset{y}{\text{min}} \; \phi(y)$.
  • Figure 3: The performance analysis of IPM-LSTM on a convex QP (RHS).
  • Figure 4: The objective values returned by IPM-LSTM at each IPM iteration on a convex QP (RHS).
  • Figure 5: The relationship between the error of the linear system solution with different parameter settings of LSTM.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Proposition 1: bellavia1998inexact
  • proof