Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An interior-point trust-region method for nonsmooth regularized bound-constrained optimization

Geoffroy Leconte, Dominique Orban

TL;DR

An interior-point method for nonsmooth regularized bound-constrained optimization problems, using a variant of the proximal quasi-Newton trust-region algorithm TR of arXiv:2103.15993v3 to solve the barrier subproblems, with additional assumptions inspired from well-known smooth interior-point trust-region methods.

Abstract

We develop an interior-point method for nonsmooth regularized bound-constrained optimization problems. Our method consists of iteratively solving a sequence of unconstrained nonsmooth barrier subproblems. We use a variant of the proximal quasi-Newton trust-region algorithm TR of arXiv:2103.15993v3 to solve the barrier subproblems, with additional assumptions inspired from well-known smooth interior-point trust-region methods. We show global convergence of our algorithm with respect to the criticality measure of arXiv:2103.15993v3. Under an additional assumption linked to the convexity of the nonsmooth term in the objective, we present an alternative interior-point algorithm with a slightly modified criticality measure, which performs better in practice. Numerical experiments show that our algorithm performs better than the trust-region method TR, the trust-region method with diagonal hessian approximations TRDH of arXiv:2309.08433, and the quadratic regularization method R2 of arXiv:2103.15993v3 for two out of four tested bound-constrained problems. On those two problems, our algorithm obtains smaller objective values than the other solvers using fewer objective and gradient evaluations. On the two other problems, it performs similarly to TR, R2 and TRDH.

An interior-point trust-region method for nonsmooth regularized bound-constrained optimization

TL;DR

An interior-point method for nonsmooth regularized bound-constrained optimization problems, using a variant of the proximal quasi-Newton trust-region algorithm TR of arXiv:2103.15993v3 to solve the barrier subproblems, with additional assumptions inspired from well-known smooth interior-point trust-region methods.

Abstract

We develop an interior-point method for nonsmooth regularized bound-constrained optimization problems. Our method consists of iteratively solving a sequence of unconstrained nonsmooth barrier subproblems. We use a variant of the proximal quasi-Newton trust-region algorithm TR of arXiv:2103.15993v3 to solve the barrier subproblems, with additional assumptions inspired from well-known smooth interior-point trust-region methods. We show global convergence of our algorithm with respect to the criticality measure of arXiv:2103.15993v3. Under an additional assumption linked to the convexity of the nonsmooth term in the objective, we present an alternative interior-point algorithm with a slightly modified criticality measure, which performs better in practice. Numerical experiments show that our algorithm performs better than the trust-region method TR, the trust-region method with diagonal hessian approximations TRDH of arXiv:2309.08433, and the quadratic regularization method R2 of arXiv:2103.15993v3 for two out of four tested bound-constrained problems. On those two problems, our algorithm obtains smaller objective values than the other solvers using fewer objective and gradient evaluations. On the two other problems, it performs similarly to TR, R2 and TRDH.
Paper Structure (15 sections, 145 equations, 4 figures, 4 tables, 3 algorithms)

This paper contains 15 sections, 145 equations, 4 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Stationarity
  4. Projected-directions methods
  5. Barrier methods
  6. Barrier subproblem
  7. Convergence of the inner iterations
  8. Convergence of the outer iterations
  9. Convergence with a new criticality measure
  10. Implementation and numerical experiments
  11. Box-constrained quadratic problem
  12. Sparse nonnegative matrix factorization (NNMF)
  13. FitzHugh-Nagumo problem (FH)
  14. Constrained basis pursuit denoise (BPDN)
  15. Discussion and future work

Figures (4)

  • Figure 1: \newlabelfig:qp-rand-plots0 Plots of the objective of \ref{['eq:constrained-qp']} per gradient evaluation with different solvers.
  • Figure 2: \newlabelfig:nnmf-plots0 Plots of the objective of \ref{['eq:nnmf']} per gradient evaluation with different solvers.
  • Figure 3: \newlabelfig:fh-plots0 Plots of the objective of \ref{['eq:fh-cstr']} per gradient evaluation with different solvers.
  • Figure 4: \newlabelfig:bpdn-plots0 Plots of the objective of \ref{['eq:bpdn-cstr']} per gradient evaluation with different solvers.

Theorems & Definitions (29)

  • Proof 1
  • Proof 2
  • Proof 3
  • Proof 4
  • Proof 5
  • Proof 6
  • Proof 7
  • Proof 8
  • Proof 9
  • Proof 10
  • ...and 19 more