Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method

Yuchen Fang, Jihun Kim, Sen Na, James Demmel, Javad Lavaei

TL;DR

A trust-region interior-point stochastic sequential quadratic programming method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints, which builds stochastic oracles at each iteration.

Abstract

In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their stochastic estimates can be constructed. In particular, at each iteration our method builds stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.

A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method

TL;DR

A trust-region interior-point stochastic sequential quadratic programming method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints, which builds stochastic oracles at each iteration.

Abstract

In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their stochastic estimates can be constructed. In particular, at each iteration our method builds stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.
Paper Structure (16 sections, 10 theorems, 86 equations, 4 figures, 1 algorithm)

This paper contains 16 sections, 10 theorems, 86 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. TR-IP-SSQP subproblem
  4. Computing the (rescaled) trial step
  5. TR-IP-SSQP Method
  6. Probabilistic oracles
  7. Algorithm design
  8. Convergence Analysis
  9. Fundamental lemmas
  10. Global convergence
  11. Merit parameter behavior
  12. Numerical Experiments
  13. Algorithm setup
  14. CUTEst
  15. Constrained Logistic regression
  16. ...and 1 more sections

Key Result

Lemma 4.3

Under Assumption assump1 and the event ${\mathcal{A}}_k$, if is satisfied, then Line 4 of Algorithm Alg:STORM will not be triggered.

Figures (4)

  • Figure 1: Relative KKT residuals of TR-IP-SSQP with different barrier parameter setups. In every plot, each box corresponds to TR-IP-SSQP with one kind of Hessian construction.
  • Figure 2: Performance Profile of TR-IP-SSQP when $\theta_k=0.9999^k$. Each trajectory represents one algorithm configuration.
  • Figure 3: Performance Profile of TR-IP-SSQP and Fully-TR-IP-SSQP when $\theta_k=0.9999^k$. Each trajectory represents one algorithm configuration.
  • Figure 4: Performance Profile of TR-IP-SSQP and Fully-TR-IP-SSQP over 40 constrained Logistic regression problem instances. Each trajectory represents one method.

Theorems & Definitions (25)

  • Definition 3.1: Probabilistic first-order oracle
  • Definition 3.2: Probabilistic zeroth-order oracle
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Lemma 4.3
  • Proof 1
  • Lemma 4.4
  • Proof 2
  • Lemma 4.5
  • ...and 15 more