Relationships between full-space and subspace quadratic interpolation models and simplex derivatives
Yiwen Chen
TL;DR
The paper addresses how to relate full-space and subspace derivative-free interpolation tools used in high-dimensional optimization. It develops explicit conversion formulas and coincidence results for MN, MFN, LFU, GSG, GSH, and QGSD under the affine-subspace assumption, showing that full-space models reduce to subspace models via a projection by $Q$ and that the subspace formulations capture the same behavior on the subspace and its orthogonal complement. These findings provide a rigorous framework for analyzing subspace approximation techniques and inform the design and analysis of derivative-free methods in high dimensions. The work also points to extensions of error bounds to subspace-sampled settings and suggests directions for comparing modeling techniques across full-space and subspace formulations.
Abstract
Quadratic interpolation models and simplex derivatives are fundamental tools in numerical optimization, particularly in derivative-free optimization. When constructed in suitably chosen affine subspaces, these tools have been shown to be especially effective for high-dimensional derivative-free optimization problems, where full-space model construction is often impractical. In this paper, we analyze the relationships between full-space and subspace formulations of these tools. In particular, we derive explicit conversion formulas between full-space and subspace models, including minimum-norm models, minimum Frobenius norm models, least Frobenius norm updating models, as well as models constructed via generalized simplex gradients and Hessians. We show that the full-space and subspace models coincide on the affine subspace and, in general, along directions in the orthogonal complement. Overall, our results provide a theoretical framework for understanding subspace approximation techniques and offer insight into the design and analysis of derivative-free optimization methods.