Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Trust-Region Algorithm for Noisy Equality Constrained Optimization

Shigeng Sun, Jorge Nocedal

TL;DR

The algorithm proposed in this paper introduces a new criterion for accepting a step and for updating the trust region that makes use of an estimate in the noise in the problem, determined by the level of noise.

Abstract

This paper introduces a modified Byrd-Omojokun (BO) trust region algorithm to address the challenges posed by noisy function and gradient evaluations. The original BO method was designed to solve equality constrained problems and it forms the backbone of some interior point methods for general large-scale constrained optimization. A key strength of the BO method is its robustness in handling problems with rank-deficient constraint Jacobians. The algorithm proposed in this paper introduces a new criterion for accepting a step and for updating the trust region that makes use of an estimate in the noise in the problem. The analysis presented here gives conditions under which the iterates converge to regions of stationary points of the problem, determined by the level of noise. This analysis is more complex than for line search methods because the trust region carries (noisy) information from previous iterates. Numerical tests illustrate the practical performance of the algorithm.

A Trust-Region Algorithm for Noisy Equality Constrained Optimization

TL;DR

The algorithm proposed in this paper introduces a new criterion for accepting a step and for updating the trust region that makes use of an estimate in the noise in the problem, determined by the level of noise.

Abstract

This paper introduces a modified Byrd-Omojokun (BO) trust region algorithm to address the challenges posed by noisy function and gradient evaluations. The original BO method was designed to solve equality constrained problems and it forms the backbone of some interior point methods for general large-scale constrained optimization. A key strength of the BO method is its robustness in handling problems with rank-deficient constraint Jacobians. The algorithm proposed in this paper introduces a new criterion for accepting a step and for updating the trust region that makes use of an estimate in the noise in the problem. The analysis presented here gives conditions under which the iterates converge to regions of stationary points of the problem, determined by the level of noise. This analysis is more complex than for line search methods because the trust region carries (noisy) information from previous iterates. Numerical tests illustrate the practical performance of the algorithm.

Paper Structure

This paper contains 14 sections, 20 theorems, 194 equations, 3 figures, 1 table.

Table of Contents

  1. Problem Statement
  2. Literature Review
  3. The Algorithm
  4. Specification of the Algorithm
  5. Global Convergence
  6. Reduction in the Feasibility Measure
  7. Feasibility Under the Full Rank Assumption
  8. Reduction in the Optimality Measure
  9. Summary of the Convergence Results
  10. Numerical Results
  11. Ability to Recover from Small Trust Region
  12. Premature Shrinkage of the Trust Region at Run Time
  13. The Cases where the Unmodified Algorithm Performs Well
  14. Final Remarks

Key Result

lemma 1

The step $p_k$ computed by Algorithm algorithmBO satisfies

Figures (3)

  • Figure 1: Testing the Byrd-Omojokun algorithm with and without noise, and the modified method.
  • Figure 2: Performance of the algorithms with a sufficiently initial trust region
  • Figure 3: Cutest Problem BYRDSPHR, Initialized with TR = $10^{-7}$

Theorems & Definitions (37)

  • lemma 1
  • lemma 2: Accuracy of the Model of the Merit Function
  • proof
  • lemma 3: Increase of the Trust Region
  • proof
  • corollary 1: Lower Bound of Trust Region Radius
  • proof
  • lemma 4: Merit Function Reduction
  • proof
  • proposition 1: Finite Time Entry to Critical Region I of Feasibility
  • ...and 27 more