Gaussian Process Bandit Optimization with Machine Learning Predictions and Application to Hypothesis Generation
Xin Jennifer Chen, Yunjin Tong
TL;DR
PA-GP-UCB addresses optimization with an expensive ground-truth feedback and a cheap, biased prediction by modeling both sources with a two-output Gaussian process and introducing a control-variates estimator to bias-correct predictions. The algorithm operates in offline and online stages, leveraging offline prediction data to tighten uncertainty and online observations to adaptively select queries, while preserving the $\tilde{O}(\sqrt{T\beta_T\gamma_T})$ regret rate up to a prediction-contingent constant. Theoretical guarantees show a strictly better leading constant when predictions are informative and the offline data provides global variance reduction, with a bound that scales as $\sqrt{C_1 \beta_T T [1-(1-R)\rho^2]\gamma_T}$. Empirically, PA-GP-UCB outperforms Vanilla GP-UCB and naïve baselines on synthetic benchmarks and a real-world hypothesis-generation task, including cases with LLM-based predictions and structure-preserving continuous embeddings, highlighting its potential for accelerated, interpretable discovery in data-scarce, costly-evaluation domains.
Abstract
Many real-world optimization problems involve an expensive ground-truth oracle (e.g., human evaluation, physical experiments) and a cheap, low-fidelity prediction oracle (e.g., machine learning models, simulations). Meanwhile, abundant offline data (e.g., past experiments and predictions) are often available and can be used to pretrain powerful predictive models, as well as to provide an informative prior. We propose Prediction-Augmented Gaussian Process Upper Confidence Bound (PA-GP-UCB), a novel Bayesian optimization algorithm that leverages both oracles and offline data to achieve provable gains in sample efficiency for the ground-truth oracle queries. PA-GP-UCB employs a control-variates estimator derived from a joint Gaussian process posterior to correct prediction bias and reduce uncertainty. We prove that PA-GP-UCB preserves the standard regret rate of GP-UCB while achieving a strictly smaller leading constant that is explicitly controlled by prediction quality and offline data coverage. Empirically, PA-GP-UCB converges faster than Vanilla GP-UCB and naive prediction-augmented GP-UCB baselines on synthetic benchmarks and on a real-world hypothesis evaluation task grounded in human behavioral data, where predictions are provided by large language models. These results establish PA-GP-UCB as a general and sample-efficient framework for hypothesis generation under expensive feedback.