On the Almost Sure Convergence of the Stochastic Three Points Algorithm

TL;DR

This work studies derivative-free optimization for differentiable, lower-bounded functions in $\mathbb{R}^d$ using the stochastic three points (STP) algorithm. It delivers the first almost-sure convergence results for STP across three function classes (smooth nonconvex, smooth convex, and smooth strongly convex), detailing explicit rates for both the best-iterate gradient norm and the final iterate. In the smooth case, the best-iterate gradient converges a.s. at $o(1/T^{1/2-\epsilon})$ for any $\epsilon\in(0,1/2)$, while for convex objectives $f(\theta^T)$ converges to the optimum with a rate $o(1/T^{1-\epsilon})$ a.s. (and $O(d/T)$ in expectation), and for strongly convex functions, directional-derivative-based STP achieves geometric convergence in expectation and almost surely with rates $O\left((1-\mu/(2\pi dL))^T\right)$ and $o\left((1-s\mu/(2\pi dL))^T\right)$ respectively. The results rely on precise step-size schedules and properties of random search directions drawn from distributions with favorable projection characteristics, and are complemented by numerical experiments showing practical competitiveness against other zeroth-order methods. Overall, the paper advances the theoretical understanding of almost-sure convergence for zeroth-order methods and clarifies the dependence on dimension $d$ in convergence rates.

Abstract

The stochastic three points (STP) algorithm is a derivative-free optimization technique designed for unconstrained optimization problems in $\mathbb{R}^d$. In this paper, we analyze this algorithm for three classes of functions: smooth functions that may lack convexity, smooth convex functions, and smooth functions that are strongly convex. Our work provides the first almost sure convergence results of the STP algorithm, alongside some convergence results in expectation. For the class of smooth functions, we establish that the best gradient iterate of the STP algorithm converges almost surely to zero at a rate of $o(1/{T^{\frac{1}{2}-ε}})$ for any $ε\in (0,\frac{1}{2})$, where $T$ is the number of iterations. Furthermore, within the same class of functions, we establish both almost sure convergence and convergence in expectation of the final gradient iterate towards zero. For the class of smooth convex functions, we establish that $f(θ^T)$ converges to $\inf_{θ\in \mathbb{R}^d} f(θ)$ almost surely at a rate of $o(1/{T^{1-ε}})$ for any $ε\in (0,1)$, and in expectation at a rate of $O(\frac{d}{T})$ where $d$ is the dimension of the space. Finally, for the class of smooth functions that are strongly convex, we establish that when step sizes are obtained by approximating the directional derivatives of the function, $f(θ^T)$ converges to $\inf_{θ\in \mathbb{R}^d} f(θ)$ in expectation at a rate of $O((1-\fracμ{2πdL})^T)$, and almost surely at a rate of $o((1-s\fracμ{2πdL})^T)$ for any $s\in (0,1)$, where $μ$ and $L$ are the strong convexity and smoothness parameters of the function.

Figures (3)

• Figure 1: Logarithmic decay of gradient norm vs. Iterations.
• Figure 2: Logarithmic Decay of gradient norm vs. Time.
• Figure 3: Convergence rate of the best gradient iterate.

Theorems & Definitions (33)

• Remark 1
• Lemma 1
• Theorem 1
• Remark 2
• Remark 3
• Theorem 2
• Theorem 3
• Remark 4
• Theorem 4
• Remark 5