Time-Varying Gaussian Process Bandits with Unknown Prior

TL;DR

The paper tackles time-varying Bayesian optimisation when the prior over the unknown function is uncertain. It introduces PE-GP-UCB, a GP-UCB-based method that jointly optimises and progressively eliminates unlikely priors, while maintaining an optimistic search over the remaining candidates. A novel regret bound is derived, showing that, with high probability, the true prior is never rejected and the cumulative regret scales with the number of priors via a $\sqrt{|\mathcal{U}|}$ factor and the maximum information gain across priors. Empirical results on synthetic and real-world time-varying tasks demonstrate that PE-GP-UCB outperforms MLE, Fully Bayesian treatment, and Regret Balancing, particularly in nonstationary settings. Overall, the work extends GP-bandits to unknown priors with provable guarantees and practical improvements in dynamic optimisation scenarios.

Abstract

Bayesian optimisation requires fitting a Gaussian process model, which in turn requires specifying prior on the unknown black-box function -- most of the theoretical literature assumes this prior is known. However, it is common to have more than one possible prior for a given black-box function, for example suggested by domain experts with differing opinions. In some cases, the type-II maximum likelihood estimator for selecting prior enjoys the consistency guarantee, but it does not universally apply to all types of priors. If the problem is stationary, one could rely on the Regret Balancing scheme to conduct the optimisation, but in the case of time-varying problems, such a scheme cannot be used. To address this gap in existing research, we propose a novel algorithm, PE-GP-UCB, which is capable of solving time-varying Bayesian optimisation problems even without the exact knowledge of the function's prior. The algorithm relies on the fact that either the observed function values are consistent with some of the priors, in which case it is easy to reject the wrong priors, or the observations are consistent with all candidate priors, in which case it does not matter which prior our model relies on. We provide a regret bound on the proposed algorithm. Finally, we empirically evaluate our algorithm on toy and real-world time-varying problems and show that it outperforms the maximum likelihood estimator, fully Bayesian treatment of unknown prior and Regret Balancing.

Figures (4)

• Figure 1: Intuition behind the prior elimination. Solid lines represent the mean functions of two models - GP1 and GP2, produced by fitting the observed points while assuming different prior, and shaded regions are confidence intervals. The point marked by the star lies in the confidence intervals of both models, whereas the point marked by the diamond lies in the one of GP1.
• Figure 2: Results on the toy problem. We ran 30 seeds and in subfigure a) we show the mean values in solid lines and standard errors as shaded regions. In subfigure b) we show the proportion of times that the methods chose each prior accross all timesteps and all seeds. The $n$th prior has the $n$th hill tallest and the prior number $0$ has all hills of equal height.
• Figure 3: Plot of mean functions of different priors used for the toy problem. All priors used the same RBF kernel. The true prior mean (the second one) is shown with a solid, black line, whereas all other prior means are shown in dashed lines.
• Figure 4: Results on the Intel temperature data. We ran 30 seeds and in subfigure a) we show the mean values in solid lines and standard errors as shaded regions. In subfigure b) we show the proportion of times that the methods chose each prior accross all timesteps and all seeds.

Theorems & Definitions (11)

• Theorem 3.1: Appendix C1 of bogunovic2016time
• Theorem 3.3: Appendix C1 of bogunovic2016time
• Theorem 4.1
• Lemma 5.0
• Lemma 5.0
• Lemma 5.0
• Lemma 6.1
• Corollary 6.2
• proof
• proof