Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Projection-based curve pattern search for black-box optimization over smooth convex sets

Xiaoxi Jia, Matteo Lapucci, Pierluigi Mansueto

TL;DR

This work addresses black-box optimization of a smooth objective over a smooth convex set by embedding the projection operator into a curvilinear polling scheme. By defining feasible search paths via projection arcs and using a fixed coordinate-direction basis, the authors derive both a gradient-based and a derivative-free algorithm that provably converge to stationary points without requiring dense direction sets. The derivative-free method (FSP) is shown to be competitive with leading projection-based approaches on standard benchmarks, especially when projections are costly. Overall, the approach offers a simple, projection-centric path to efficient constrained black-box optimization with solid theoretical guarantees and practical performance.

Abstract

In this paper, we deal with the problem of optimizing a black-box smooth function over a full-dimensional smooth convex set. We study sets of feasible curves that allow to properly characterize stationarity of a solution and possibly carry out sound backtracking curvilinear searches. We then propose a general pattern search algorithmic framework that exploits curves of this type to carry out poll steps and for which we prove properties of asymptotic convergence to stationary points. We particularly point out that the proposed framework covers the case where search curves are arcs induced by the Euclidean projection of coordinate directions. The method is finally proved to arguably be superior, on smooth problems, than other recent projection-based algorithms and is competitive with state-of-the-art methods from the literature on constrained black-box optimization.

Projection-based curve pattern search for black-box optimization over smooth convex sets

TL;DR

This work addresses black-box optimization of a smooth objective over a smooth convex set by embedding the projection operator into a curvilinear polling scheme. By defining feasible search paths via projection arcs and using a fixed coordinate-direction basis, the authors derive both a gradient-based and a derivative-free algorithm that provably converge to stationary points without requiring dense direction sets. The derivative-free method (FSP) is shown to be competitive with leading projection-based approaches on standard benchmarks, especially when projections are costly. Overall, the approach offers a simple, projection-centric path to efficient constrained black-box optimization with solid theoretical guarantees and practical performance.

Abstract

In this paper, we deal with the problem of optimizing a black-box smooth function over a full-dimensional smooth convex set. We study sets of feasible curves that allow to properly characterize stationarity of a solution and possibly carry out sound backtracking curvilinear searches. We then propose a general pattern search algorithmic framework that exploits curves of this type to carry out poll steps and for which we prove properties of asymptotic convergence to stationary points. We particularly point out that the proposed framework covers the case where search curves are arcs induced by the Euclidean projection of coordinate directions. The method is finally proved to arguably be superior, on smooth problems, than other recent projection-based algorithms and is competitive with state-of-the-art methods from the literature on constrained black-box optimization.

Paper Structure

This paper contains 11 sections, 10 theorems, 12 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Characterization of the feasible set
  3. Searching along curves
  4. The algorithmic scheme
  5. First-order method
  6. Derivative-free method
  7. Numerical results
  8. First set of problems
  9. Cutest problems
  10. Conclusions
  11. Proof of Proposition \ref{['prop:conv_fo']}

Key Result

Proposition 2.1

A point $\bar{x}\in C$ is a stationary point in the sense of eq:stat_tangent_cone for problem Eq:P if and only if we have

Figures (5)

  • Figure 1: Visualization of the construction of case $x\in\partial C$ of the proof of Proposition \ref{['prop:veloc_canon']}.
  • Figure 2: Search curves implicitly defined by the coordinate directions and projection.
  • Figure 3: Performance profiles in terms of $n_f$ and $n_p$ obtained by FSP and PPM on the Cutest problems listed in Table \ref{['tab::problems']} under both types of unit hyper-sphere constraints, i.e., $c = [0,\ldots,0]^T$ and $c = [5,\ldots,5]^T$.
  • Figure 4: Data profiles obtained by FSP, SPG-DDS-proj, SPG-DDS-barrier and DDS-proj for tolerances $\varepsilon \in \{10^{-2}, 10^{-4}, 10^{-6}, 10^{-8}\}$ on the Cutest problems listed in Table \ref{['tab::problems']}, under unit hyper-sphere constraints with both $c = [0,\ldots,0]^T$ and $c = [5,\ldots,5]^T$.
  • Figure 5: Data profiles obtained by FSP and ORTHOMADS for tolerances $\varepsilon \in \{10^{-2}, 10^{-4}, 10^{-6}, 10^{-8}\}$ on the Cutest problems listed in Table \ref{['tab::problems']}, under unit hyper-sphere constraints with both $c = [0,\ldots,0]^T$ and $c = [5,\ldots,5]^T$.

Theorems & Definitions (12)

  • Proposition 2.1
  • Proposition 2.2
  • Definition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Lemma 3.4
  • Proposition 3.5
  • Proposition 4.1
  • Proposition 4.2
  • Lemma 4.3
  • ...and 2 more