Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need

TL;DR

The paper tackles the computational bottleneck of multi-step look-ahead Bayesian optimization caused by nested Monte Carlo in acquisition-function evaluation. It introduces Multilevel Monte Carlo (MLMC) to construct telescoping estimators that couple inexpensive coarse simulations with accurate fine simulations, achieving the canonical $O( ext{ε}^{-2})$ cost for nested MC and mitigating dimension dependence. The authors provide a rigorous decomposition of SAA error into variance and bias, establish MLMC estimators for both the acquisition function and its maximizer, and prove convergence rates with improved costs; they also show that antithetic coupling can boost variance decay to $eta \,\approx \,1.5$, further improving efficiency. Numerical experiments on a 1D toy problem and BO benchmarks confirm substantial reductions in computational effort for a given accuracy and illustrate practical implementation considerations. The work lays foundations for extensions to multi-index MC, randomized MLMC, and quasi-Monte Carlo variants within Bayesian optimization.

Abstract

We leverage multilevel Monte Carlo (MLMC) to improve the performance of multi-step look-ahead Bayesian optimization (BO) methods that involve nested expectations and maximizations. Often these expectations must be computed by Monte Carlo (MC). The complexity rate of naive MC degrades for nested operations, whereas MLMC is capable of achieving the canonical MC convergence rate for this type of problem, independently of dimension and without any smoothness assumptions. Our theoretical study focuses on the approximation improvements for twoand three-step look-ahead acquisition functions, but, as we discuss, the approach is generalizable in various ways, including beyond the context of BO. Our findings are verified numerically and the benefits of MLMC for BO are illustrated on several benchmark examples. Code is available at https://github.com/Shangda-Yang/MLMCBO .

Figures (10)

• Figure 1: A graphical description of MLMC complexity (green area) improvement over nested MC (red area).
• Figure 2: Panel \ref{['fig:1DToy']}: (i) The blue solid line is the objective function $g$ (with axis on the right). The function has a unique global maximizer and maximum. (ii) The black solid line is the analytical EI acquisition function, and the black dot is the reference solution. (iii) The dashed colored lines are the Monte Carlo approximation of the acquisition function with varying $N$, and the corresponding dots are the respective maximums. Low-accuracy (small $N$) approximations can result in maximizer of the approximation to be far from the true maximizer. Panel \ref{['fig:1DMSE']}: Complexity diagram of MLMC and nested MC approximation of two-step look-ahead EI with the cost measured by the number of operations. The reference solution for MSE is computed with high accuracy. Each curve is computed with 200 realizations.
• Figure 3: Convergence of the BO algorithm with respect to the cumulative wall time in seconds, with error bars (computed with 20 realizations). The Matérn kernel is applied. The initial BO run starts with $2\times d$ observations.
• Figure 4: Sample average approximation rate of convergence. The Matern kernel is applied with six observations. \ref{['fig:OneEIOuter']}: convergence with respect to $N$. Rate of regression: -0.92. \ref{['fig:OneEIInner']}: convergence with respect to $M$ with fixed $N_l = 2^{5}$. Rate of regression: -1.05. 100 realizations are used for both plots.
• Figure 5: Complexity of MLMC value function and the corresponding optimizer. \ref{['fig:MLVal']}: a fitted slope of -1.08. \ref{['fig:optMLVal']}: a fitted slope of -1.16. 200 realizations are used for both plots.

Theorems & Definitions (24)

• Proposition 1: Theorem 12 of kim2015guide
• Proposition 2
• Proposition 3
• Theorem 1: Q-Function Convergence
• Theorem 2
• Remark 1
• proof
• Corollary 1: Value Function Convergence
• Corollary 2: Maximizer Convergence
• Remark 2