Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A structured modified Newton approach for solving systems of nonlinear equations arising in interior-point methods for quadratic programming

David Ek, Anders Forsgren

TL;DR

This work proposes a structured modified Newton approach where each modified Jacobian is composed of a previous Jacobian, plus one low-rank update matrix per succeeding iteration, and preserves the nonzero pattern of the Jacobian.

Abstract

The focus in this work is on interior-point methods for inequality-constrained quadratic programs, and particularly on the system of nonlinear equations to be solved for each value of the barrier parameter. Newton iterations give high quality solutions, but we are interested in modified Newton systems that are computationally less expensive at the expense of lower quality solutions. We propose a structured modified Newton approach where each modified Jacobian is composed of a previous Jacobian, plus one low-rank update matrix per succeeding iteration. Each update matrix is, for a given rank, chosen such that the distance to the Jacobian at the current iterate is minimized, in both 2-norm and Frobenius norm. The approach is structured in the sense that it preserves the nonzero pattern of the Jacobian. The choice of update matrix is supported by results in an ideal theoretical setting. We also produce numerical results with a basic interior-point implementation to investigate the practical performance within and beyond the theoretical framework. In order to improve performance beyond the theoretical framework, we also motivate and construct two heuristics to be added to the method.

A structured modified Newton approach for solving systems of nonlinear equations arising in interior-point methods for quadratic programming

TL;DR

This work proposes a structured modified Newton approach where each modified Jacobian is composed of a previous Jacobian, plus one low-rank update matrix per succeeding iteration, and preserves the nonzero pattern of the Jacobian.

Abstract

The focus in this work is on interior-point methods for inequality-constrained quadratic programs, and particularly on the system of nonlinear equations to be solved for each value of the barrier parameter. Newton iterations give high quality solutions, but we are interested in modified Newton systems that are computationally less expensive at the expense of lower quality solutions. We propose a structured modified Newton approach where each modified Jacobian is composed of a previous Jacobian, plus one low-rank update matrix per succeeding iteration. Each update matrix is, for a given rank, chosen such that the distance to the Jacobian at the current iterate is minimized, in both 2-norm and Frobenius norm. The approach is structured in the sense that it preserves the nonzero pattern of the Jacobian. The choice of update matrix is supported by results in an ideal theoretical setting. We also produce numerical results with a basic interior-point implementation to investigate the practical performance within and beyond the theoretical framework. In order to improve performance beyond the theoretical framework, we also motivate and construct two heuristics to be added to the method.

Paper Structure

This paper contains 15 sections, 11 theorems, 73 equations, 2 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation
  3. Background
  4. A structured modified Newton approach
  5. At a general iteration $k$
  6. Convergence
  7. Reduced systems
  8. Compatibility with previous work on interior-point methods
  9. Implementation
  10. Basic algorithm
  11. Benchmark problems
  12. Heuristics
  13. Numerical results
  14. Conclusion
  15. Appendix

Key Result

Lemma 2.1

Under Assumption ass1 there exists $\delta>0$ such that $F'(z)$ is continuous and nonsingular for $z \in \mathcal{B}(z^*, \delta)$ and for some constant $M>0$.

Figures (2)

  • Figure 1: The left-hand side shows step sizes and convergence on benchmark problem qafiro. The right-hand side shows the inverse of the limiting step sizes and the relative error in the search direction for negative components of the modified Newton direction in mN-r(2), at the iteration marked by the red circle.
  • Figure 2: Step sizes and convergence for mN-r($r$), $r = [2, 4]$, combined with heuristic H1 and H2 on benchmark problem qafiro.

Theorems & Definitions (13)

  • Definition 2.1: Order-notation
  • Definition 2.2: Neighborhood
  • Lemma 2.1
  • Lemma 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Proposition 3.1
  • Lemma 3.1
  • Theorem 3.1
  • Proposition 3.2
  • ...and 3 more