CAGES: Cost-Aware Gradient Entropy Search for Efficient Local Multi-Fidelity Bayesian Optimization

TL;DR

The paper tackles efficient local Bayesian optimization in high-dimensional spaces when multiple cheaper information sources are available. It introduces CAGES, which couples latent-variable Gaussian processes for flexible cross-source modeling with a cost-aware gradient-entropy acquisition that seeks to maximize gradient information per cost. Key contributions include a LVGP MIS model, a gradient-entropy based, cost-normalized acquisition, and a practical two-loop optimization algorithm. Empirical results on Rosenbrock and CartPole-v1 show that CAGES outperforms single-source BO and existing multi-fidelity baselines, indicating strong potential for fast, high-dimensional optimization in RL and simulation-intensive domains.

Abstract

Bayesian optimization (BO) is a popular approach for optimizing expensive-to-evaluate black-box objective functions. An important challenge in BO is its application to high-dimensional search spaces due in large part to the curse of dimensionality. One way to overcome this challenge is to focus on local BO methods that aim to efficiently learn gradients, which have shown strong empirical performance on high-dimensional problems including policy search in reinforcement learning (RL). Current local BO methods assume access to only a single high-fidelity information source whereas, in many problems, one has access to multiple cheaper approximations of the objective. We propose a novel algorithm, Cost-Aware Gradient Entropy Search (CAGES), for local BO of multi-fidelity black-box functions. CAGES makes no assumption about the relationship between different information sources, making it more flexible than other multi-fidelity methods. It also employs a new information-theoretic acquisition function, which enables systematic identification of samples that maximize the information gain about the unknown gradient per evaluation cost. We demonstrate CAGES can achieve significant performance improvements compared to other state-of-the-art methods on synthetic and benchmark RL problems.

Figures (2)

• Figure 1: Best found objective value versus total cost for the Rosenbrock benchmark averaged over 10 replicates.
• Figure 2: Best found reward value (negative objective) versus total cost for the Cartpole-v1 RL problem averaged over 10 replicates.

Theorems & Definitions (2)

• Remark 1
• Remark 2