Expected Improvement via Gradient Norms

TL;DR

EI-GN augments the standard EI acquisition by applying the improvement principle to a gradient-aware auxiliary objective, reducing EI's tendency to over-exploit local optima. By modeling $f$ and $ abla f$ with independent Gaussian Processes, the method derives a tractable acquisition $ ext{EI-GN}(oldsymbol{x}) = ext{EI}_f(oldsymbol{x}) - oldsymbol{ abla f}$-term via a mean-field approximation, yielding a strong gradient-informed exploration signal. Empirically, EI-GN delivers consistent gains across synthetic multimodal benchmarks, GP-sampled objectives, and policy-search problems, especially where standard EI stalls in low-improvement regions. The approach preserves EI’s global improvement structure while injecting a principled stationarity bias, offering robust performance with scalable computation and practical applicability to gradient-enabled BO tasks.

Abstract

Bayesian Optimization (BO) is a principled approach for optimizing expensive black-box functions, with Expected Improvement (EI) being one of the most widely used acquisition functions. Despite its empirical success, EI is known to be overly exploitative and can converge to suboptimal stationary points. We propose Expected Improvement via Gradient Norms (EI-GN), a novel acquisition function that applies the improvement principle to a gradient-aware auxiliary objective, thereby promoting sampling in regions that are both high-performing and approaching first-order stationarity. EI-GN relies on gradient observations used to learn gradient-enhanced surrogate models that enable principled gradient inference from function evaluations. We derive a tractable closed-form expression for EI-GN that allows efficient optimization and show that the proposed acquisition is consistent with the improvement-based acquisition framework. Empirical evaluations on standard BO benchmarks demonstrate that EI-GN yields consistent improvements against standard baselines. We further demonstrate applicability of EI-GN to control policy learning problems.

Figures (11)

• Figure 1: Five largest solutions found by EI (cyan) and the proposed acquisition function EI-GN (red) for the Holder Table function in $\mathbf{x} \in [-10, 10]^2$. EI gets trapped in one of the many local maxima and repeatedly suggests the same query, whereas EI-GN undergoes informed exploration to identify various solutions with higher objective values.
• Figure 2: Comparison of acquistion behavior for a univariate mixture of Gaussians with a wide local basin and a narrow basin containing the global maximum in $\mathbf{x} \in [0, 1]$. Top: Objective landscape and sampled data for GP learning. Bottom: Acquisition values for EI (cyan) and EI-GN (red) with regions corresponding to large values shaded.
• Figure 3: Results displaying mean $\pm$ one standard error for synthetic benchmarks: Shekel (4$d$), Hartmann (6$d$), Cosine (8$d$), Griewank (10$d$), and Ackley (14$d$) for EI-GN against a variety of baselines.
• Figure 4: Results displaying mean $\pm$ one standard error for within-model comparisons in 7$d$, 8$d$, and 9$d$.
• Figure 5: Results displaying mean $\pm$ one standard error for out-of-model comparisons in 7$d$, 8$d$, and 9$d$.

Theorems & Definitions (2)

• Lemma 1.1
• proof