An informational approach to the global optimization of expensive-to-evaluate functions

TL;DR

The paper tackles global optimization with expensive-to-evaluate functions by combining Kriging surrogates with an information-theoretic criterion. It introduces minimizer entropy and the IAGO algorithm within a SUR framework to sequentially choose evaluation points, demonstrating substantial evaluation savings over EI-based methods and extending to robust, noisy settings. Through conditional simulations and a probabilistic view of the global minimizers, IAGO achieves more global exploration and faster uncertainty reduction, with practical relevance for engineering problems where each evaluation is costly. The work also discusses robustness to covariance-parameter uncertainty and outlines avenues for constrained optimization and higher-dimensional applications, highlighting the method's potential for efficient, uncertainty-aware design under noise.

Abstract

In many global optimization problems motivated by engineering applications, the number of function evaluations is severely limited by time or cost. To ensure that each evaluation contributes to the localization of good candidates for the role of global minimizer, a sequential choice of evaluation points is usually carried out. In particular, when Kriging is used to interpolate past evaluations, the uncertainty associated with the lack of information on the function can be expressed and used to compute a number of criteria accounting for the interest of an additional evaluation at any given point. This paper introduces minimizer entropy as a new Kriging-based criterion for the sequential choice of points at which the function should be evaluated. Based on \emph{stepwise uncertainty reduction}, it accounts for the informational gain on the minimizer expected from a new evaluation. The criterion is approximated using conditional simulations of the Gaussian process model behind Kriging, and then inserted into an algorithm similar in spirit to the \emph{Efficient Global Optimization} (EGO) algorithm. An empirical comparison is carried out between our criterion and \emph{expected improvement}, one of the reference criteria in the literature. Experimental results indicate major evaluation savings over EGO. Finally, the method, which we call IAGO (for Informational Approach to Global Optimization) is extended to robust optimization problems, where both the factors to be tuned and the function evaluations are corrupted by noise.

Figures (12)

• Figure 1: Naive approach to optimization using Kriging: (top) prediction $\hat{f}$ (bold line) of the true function $f$ (dotted line, supposedly unknown) obtained from an initial design materialized by squares; (bottom) prediction after seven iterations of a direct minimization of $\hat{f}$.
• Figure 2: EI approach to optimization using Kriging: (top) $\hat{f}$ (bold line), 95% confidence intervals computed using $\hat{\sigma}$ (dashed line) and true function $f$ (dotted line); (bottom) expected improvement.
• Figure 3: Conditioning a simulation: (top) unknown real curve $f$ (doted line), sample points (squares) and associated Kriging prediction $\hat{f}$ (bold line); (middle) non-conditional simulation $z$, sample points and associated Kriging prediction $\hat{z}$ (bold line); (bottom) the simulation of the Kriging error $z-\hat{z}$ is picked up from the non-conditional simulation and added to the Kriging prediction to get the conditional simulation (thin line).
• Figure 4: Estimation of the conditional point mass distribution of $\bm{X}^{\ast}_{\mathbb{G}}$: (top) Kriging interpolation, 95% confidence intervals and sample points; (bottom) estimated point mass distribution of $\bm{X}^{\ast}_{\mathbb{G}}$ using 10000 conditional simulations of $F$ and a regular grid for $\mathbb{G}$.
• Figure 5: Example of prediction by Kriging (bold line) of noisy measurements represented by squares. Dashed lines represent 95% confidence regions for the prediction and the thin solid line is an example of conditional simulation obtained using the method presented in Section \ref{['sec:noise']} (crosses represent the simulated measurements.)