Complexity results and active-set identification of a derivative-free method for bound-constrained problems

TL;DR

This work analyzes a derivative-free line-search method tailored for bound-constrained optimization with a black-box objective. It combines coordinate-directed searches with a line-search that includes an extrapolation phase, establishing global convergence to stationary points via a criticality measure $\chi(x)$ and deriving worst-case complexity bounds $O(n\epsilon^{-2})$ iterations and $O(n^2\epsilon^{-2})$ function evaluations to reach $\chi(x)\le\epsilon$. It also proves finite identification of active constraints under strict complementarity, leveraging the extrapolation mechanism to hit bounds near stationary points. The results position the proposed method on par with established direct-search and model-based derivative-free approaches in terms of worst-case efficiency, while also guaranteeing finite active-set identification, which can improve practical performance by revealing the active constraint surface early in the optimization process.

Abstract

In this paper, we analyze a derivative-free line search method designed for bound-constrained problems. Our analysis demonstrates that this method exhibits a worst-case complexity comparable to other derivative-free methods for unconstrained and linearly constrained problems. In particular, when minimizing a function with $n$ variables, we prove that at most ${\cal O(nε^{-2})}$ iterations are needed to drive a criticality measure below a predefined threshold $ε$, requiring at most ${\cal O(n^2ε^{-2})}$ function evaluations. We also show that the total number of iterations where the criticality measure is not below $ε$ is upper bounded by ${\cal O(n^2ε^{-2})}$. Moreover, we investigate the method capability to identify active constraints at the final solutions. We show that, after a finite number of iterations, all the active constraints satisfying the strict complementarity condition are correctly identified.

Theorems & Definitions (38)

• Definition 1
• Definition 2
• Proposition 1: kolda:03
• Proposition 2: kolda:03
• Lemma 1
• proof
• Proposition 3
• proof
• Theorem 1
• proof