Introduction to Interpolation-Based Optimization
Lindon Roberts
TL;DR
This paper presents interpolation-based optimization (IBO) as a principled, derivative-free local optimization framework that replaces gradient-based Taylor models with interpolation-based surrogates within a trust-region setting. It develops the theory and algorithms for fully linear and fully quadratic interpolation models, derives worst-case convergence and complexity results for both first- and second-order optimality, and explains how to construct and manage interpolation sets to ensure model accuracy. The work extends IBO to constrained and noisy settings, introducing feasible interpolation models, poisedness concepts, and self-correcting geometry to maintain accuracy while remaining within feasible regions or handling bounded/noisy evaluations. Collectively, these results provide a rigorous foundation for practical IBO methods, showing how to achieve reliable convergence guarantees while balancing the costs of evaluating expensive, black-box objective functions.
Abstract
The field of derivative-free optimization (DFO) studies algorithms for nonlinear optimization that do not rely on the availability of gradient or Hessian information. It is primarily designed for settings when functions are black-box, expensive to evaluate and/or noisy. A widely used and studied class of DFO methods for local optimization is interpolation-based optimization (IBO), also called model-based DFO, where the general principles from derivative-based nonlinear optimization algorithms are followed, but local Taylor-type approximations are replaced with alternative local models constructed by interpolation. This document provides an overview of the basic algorithms and analysis for IBO, covering worst-case complexity, approximation theory for polynomial interpolation models, and extensions to constrained and noisy problems.