An optimally fast objective-function-free minimization algorithm using random subspaces
S. Bellavia, S. Gratton, B. Morini, Ph. L. Toint
TL;DR
This work introduces SKOFFAR$p$, an objective-function-free adaptive-regularisation method that minimizes nonconvex functions in randomly chosen subspaces. By sketching the $p$-th order regularised model and reconstructing full steps, it preserves convergence guarantees and attains the optimal evaluation complexity $O\left(\epsilon^{-{(p+1)/p}}\right)$ for first-order critical points, with a second-order variant achieving $O\left(\epsilon^{-2}\right)$. The analysis combines random-subspace embeddings with adaptive regularisation, showing that randomness can reduce per-iteration cost without compromising global guarantees, particularly for problems with low-rank Hessians. Numerical experiments on CUTEst problems illustrate substantial practical gains when using second-order models in low-rank settings and highlight competitive performance against established first-order OFFO methods.
Abstract
An algorithm for unconstrained non-convex optimization is described, which does not evaluate the objective function and in which minimization is carried out, at each iteration, within a randomly selected subspace. It is shown that this random approximation technique does not affect the method's convergence nor its evaluation complexity for the search of an $ε$-approximate first-order critical point, which is $\mathcal{O}(ε^{-(p+1)/p})$, where $p$ is the order of derivatives used. A variant of the algorithm using approximate Hessian matrices is also analysed and shown to require at most $\mathcal{O}(ε^{-2})$ evaluations. Preliminary numerical tests show that the random-subspace technique can significantly improve performance when used with $p=2$ in the correct context, making it very competitive when compared to standard first-order algorithms.