Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring an Alternative Line-Search Method for Lagrange-Newton Optimization

Ralf Möller

TL;DR

This work tackles the challenge of performing line searches in Newton-based optimization when equality constraints are present, which makes all stationary points saddles and renders objective-value line searches ineffective. The authors introduce a divergence-based criterion $\tau(\mathbf{x})$ for the Newton-step field and its squared valley $\check{\tau}(\mathbf{x})$, then develop a three-phase zigzag line-search strategy to traverse ravine networks that lead to stationary points. They provide a mathematical derivation of the criterion, discuss pullback directions, and present 2D experiments (with Rosenbrock, Himmelblau, Henon–Heiles, and others) plus a higher-dimensional example, illustrating the ravine structure and the method's potential and limitations. While the approach can reach saddle points and dampen step length in tight valleys, its computational cost is substantial and performance is inconsistent across problems, suggesting the need for further refinement or alternative criteria. Overall, the paper offers a novel, geometry-inspired line-search paradigm for Lagrange-Newton methods with equality constraints and lays groundwork for future improvements in ravine-traversing strategies.

Abstract

In the Lagrange-Newton method, where Newton's method is applied to a Lagrangian function that includes equality constraints, all stationary points are saddle points. It is therefore not possible to use a line-search method based on the value of the objective function; instead, the line search can operate on merit functions. In this report, we explore an alternative line-search method which is applicable to this case; it particulary addresses the damping of the step length in tight valleys. We propose a line-search criterion based on the divergence of the field of Newton step vectors. The visualization of the criterion for two-dimensional test functions reveals a network of ravines with flat bottom at the zero points of the criterion. The ravines are typically connected to stationary points. To traverse this ravine network in order to approach a stationary point, a zigzag strategy is devised. Numerical experiments demonstrate that the novel line-search strategy succeeds from most starting points in all test functions, but only exhibits the desired damping of the step length in some situations. At the present stage it is therefore difficult to appraise the utility of this contribution.

Exploring an Alternative Line-Search Method for Lagrange-Newton Optimization

TL;DR

This work tackles the challenge of performing line searches in Newton-based optimization when equality constraints are present, which makes all stationary points saddles and renders objective-value line searches ineffective. The authors introduce a divergence-based criterion for the Newton-step field and its squared valley , then develop a three-phase zigzag line-search strategy to traverse ravine networks that lead to stationary points. They provide a mathematical derivation of the criterion, discuss pullback directions, and present 2D experiments (with Rosenbrock, Himmelblau, Henon–Heiles, and others) plus a higher-dimensional example, illustrating the ravine structure and the method's potential and limitations. While the approach can reach saddle points and dampen step length in tight valleys, its computational cost is substantial and performance is inconsistent across problems, suggesting the need for further refinement or alternative criteria. Overall, the paper offers a novel, geometry-inspired line-search paradigm for Lagrange-Newton methods with equality constraints and lays groundwork for future improvements in ravine-traversing strategies.

Abstract

In the Lagrange-Newton method, where Newton's method is applied to a Lagrangian function that includes equality constraints, all stationary points are saddle points. It is therefore not possible to use a line-search method based on the value of the objective function; instead, the line search can operate on merit functions. In this report, we explore an alternative line-search method which is applicable to this case; it particulary addresses the damping of the step length in tight valleys. We propose a line-search criterion based on the divergence of the field of Newton step vectors. The visualization of the criterion for two-dimensional test functions reveals a network of ravines with flat bottom at the zero points of the criterion. The ravines are typically connected to stationary points. To traverse this ravine network in order to approach a stationary point, a zigzag strategy is devised. Numerical experiments demonstrate that the novel line-search strategy succeeds from most starting points in all test functions, but only exhibits the desired damping of the step length in some situations. At the present stage it is therefore difficult to appraise the utility of this contribution.
Paper Structure (52 sections, 1 theorem, 71 equations, 26 figures)

This paper contains 52 sections, 1 theorem, 71 equations, 26 figures.

Table of Contents

  1. Introduction
  2. Notation
  3. Newton's Method and Line Search
  4. Divergence-based Criterion
  5. Line-Search Method
  6. Zigzag Strategy
  7. Pullback Directions
  8. Approximated Gradient
  9. Singularity Curves
  10. Lagrange-Newton Method
  11. Notation
  12. Lagrangian
  13. Bordered Hessian
  14. Proof of Saddle Points
  15. Experiments on 2D Test Functions
  16. ...and 37 more sections

Key Result

Lemma 1

Given $n$: $1 \leq n$ and $\mathbf{H}$: symmetric, $n \times n$; $\bar{\mathbf{H}}$: symmetric, $n \times n$; $\mathbf{x}$: arbitrary, $n \times 1$; $\bar{\mathbf{x}}$: arbitrary, $n \times 1$; $\mathbf{0}_{n}$: zero, $n \times 1$; $f$: scalar, $1 \times 1$. Let $\bar{\mathbf{H}} = \mathbf{H}\mleft(

Figures (26)

  • Figure 1:
  • Figure 2:
  • Figure 3:
  • Figure 4:
  • Figure 5:
  • ...and 21 more figures

Theorems & Definitions (3)

  • Lemma 1
  • proof
  • proof