Depth-first directional search for nonconvex optimization

TL;DR

The paper addresses global optimization of continuous nonconvex functions over a compact convex set using derivative-free methods. It introduces DFDS, a depth-first directional search that exhaustively performs line searches along randomly drawn directions, and analyzes its performance through a geometric framework based on spherical caps, proving convergence in probability and giving an explicit complexity bound. Theoretical results show ε-accuracy can be achieved with an expected number of function evaluations scaling as $O\left(\left(\frac{2(D_0+R_\varepsilon)}{\sqrt{3}R_\varepsilon}\right)^N \frac{D_0}{R_\varepsilon} \sqrt{N} \frac{1}{\varepsilon}\right)$, and numerical experiments on benchmark problems demonstrate superior accuracy and scalability of DFDS relative to PRS and IHR under the same evaluation budget. The work provides a novel geometric lens for analyzing directional search methods and highlights DFDS’s practical potential for high-dimensional global optimization.

Abstract

Random search methods are widely used for global optimization due to their theoretical generality and implementation simplicity. This paper proposes a depth-first directional search (DFDS) algorithm for globally solving nonconvex optimization problems. Motivated by the penetrating beam of a searchlight, DFDS performs a complete stepping line search along each sampled direction before proceeding to the next, contrasting with existing directional search methods that prioritize broad exploratory coverage. We establish the convergence and computational complexity of DFDS through a novel geometric framework that models the success probability of finding a global optimizer as the surface area of a spherical cap. Numerical experiments on benchmark problems demonstrate that DFDS achieves significantly higher accuracy in locating the global optimum compared to other random search methods under the same function evaluation budget.

Figures (3)

• Figure 1: $R(x_k,d)$ when $\langle x^*-x_k,d\rangle=\alpha$ and $\overline{B}(x^*,R_\epsilon)$ ($N=2$)
• Figure 2: The spherical cap $\mathcal{S}_{\alpha}^2$ of the unit ball in $\mathbb{R}^3$
• Figure 3: The explicit value and the lower bound of $\log_{10} p_{N,\alpha}$ given in Theorem \ref{['theo p']} ($\alpha=\frac{\pi}{4}$)

Theorems & Definitions (6)

• proof
• proof
• proof
• proof
• proof
• proof