Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time-varying Gaussian Process Bandit Optimization with Experts: no-regret in logarithmically-many side queries

Eliabelle Mauduit, Eloïse Berthier, Andrea Simonetto

TL;DR

This work addresses Bayesian optimization under time-varying rewards by modeling the objective as a sequence of RKHS-valued functions with bounded drift. A novel uncertainty-injection approach recasts the problem as a heteroscedastic GP regression and introduces SparQ-GP-UCB, which refreshes stale data via logarithmically many side queries to an expert and uses sparse inference based on M-DPP to keep computations tractable. The authors prove a sublinear dynamic regret bound R_T = \\tilde{\\mathcal{O}}(\\sqrt{T}) with per-step queries Q_T = \\tilde{\\mathcal{O}}(1) and a total computational cost of \\tilde{\\mathcal{O}}(T^2), demonstrating that minimal additional information suffices to counteract temporal drift. Empirical results on synthetic and real data show SparQ-GP-UCB outperforming time-varying GP baselines, confirming practical effectiveness in tracking moving optima while maintaining scalable computation.

Abstract

We study a time-varying Bayesian optimization problem with bandit feedback, where the reward function belongs to a Reproducing Kernel Hilbert Space (RKHS). We approach the problem via an upper-confidence bound Gaussian Process algorithm, which has been proven to yield no-regret in the stationary case. The time-varying case is more challenging and no-regret results are out of reach in general in the standard setting. As such, we instead tackle the question of how many additional observations asked to an expert are required to regain a no-regret property. To do so, we formulate the presence of past observation via an uncertainty injection procedure, and we reframe the problem as a heteroscedastic Gaussian Process regression. In addition, to achieve a no-regret result, we discard long outdated observations and replace them with updated (possibly very noisy) ones obtained by asking queries to an external expert. By leveraging and extending sparse inference to the heteroscedastic case, we are able to secure a no-regret result in a challenging time-varying setting with only logarithmically-many side queries per time step. Our method demonstrates that minimal additional information suffices to counteract temporal drift, ensuring efficient optimization despite time variation.

Time-varying Gaussian Process Bandit Optimization with Experts: no-regret in logarithmically-many side queries

TL;DR

This work addresses Bayesian optimization under time-varying rewards by modeling the objective as a sequence of RKHS-valued functions with bounded drift. A novel uncertainty-injection approach recasts the problem as a heteroscedastic GP regression and introduces SparQ-GP-UCB, which refreshes stale data via logarithmically many side queries to an expert and uses sparse inference based on M-DPP to keep computations tractable. The authors prove a sublinear dynamic regret bound R_T = \\tilde{\\mathcal{O}}(\\sqrt{T}) with per-step queries Q_T = \\tilde{\\mathcal{O}}(1) and a total computational cost of \\tilde{\\mathcal{O}}(T^2), demonstrating that minimal additional information suffices to counteract temporal drift. Empirical results on synthetic and real data show SparQ-GP-UCB outperforming time-varying GP baselines, confirming practical effectiveness in tracking moving optima while maintaining scalable computation.

Abstract

We study a time-varying Bayesian optimization problem with bandit feedback, where the reward function belongs to a Reproducing Kernel Hilbert Space (RKHS). We approach the problem via an upper-confidence bound Gaussian Process algorithm, which has been proven to yield no-regret in the stationary case. The time-varying case is more challenging and no-regret results are out of reach in general in the standard setting. As such, we instead tackle the question of how many additional observations asked to an expert are required to regain a no-regret property. To do so, we formulate the presence of past observation via an uncertainty injection procedure, and we reframe the problem as a heteroscedastic Gaussian Process regression. In addition, to achieve a no-regret result, we discard long outdated observations and replace them with updated (possibly very noisy) ones obtained by asking queries to an external expert. By leveraging and extending sparse inference to the heteroscedastic case, we are able to secure a no-regret result in a challenging time-varying setting with only logarithmically-many side queries per time step. Our method demonstrates that minimal additional information suffices to counteract temporal drift, ensuring efficient optimization despite time variation.
Paper Structure (24 sections, 12 theorems, 95 equations, 2 figures, 3 algorithms)

This paper contains 24 sections, 12 theorems, 95 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Contributions
  4. Problem setting
  5. Uncertainty injection
  6. Sparse inference
  7. SparQ-GP-UCB Algorithm
  8. Main results
  9. Role of the expert
  10. Numerical results
  11. Synthetic data
  12. Real data
  13. Proofs of main theorems and additional results
  14. Proof of Theorem \ref{['Thm:regret']}
  15. Regret proof
  16. ...and 9 more sections

Key Result

proposition thmcounterproposition

Consider the problem of estimating an unknown function $f: \mathcal{D} \to \mathbb{R}$ via $T$ noisy observations, $y_t = f(x_t) + \epsilon_t$, $\epsilon_t \sim \mathcal{N}(0, \sigma^2)$ acquired at i.i.d. training inputs $X_T$. Let $f$ be a sample path of a Gaussian Process with zero mean and kerne

Figures (2)

  • Figure 1: Average regret of GP-UCB variants in the time-varying setting.
  • Figure 2: Numerical performances of GP-UCB variants on real data.

Theorems & Definitions (16)

  • proposition thmcounterproposition: JMLR:v21:19-1015
  • theorem thmcountertheorem
  • theorem thmcountertheorem
  • proposition thmcounterproposition
  • proof
  • proposition thmcounterproposition
  • proof
  • proposition thmcounterproposition
  • definition thmcounterdefinition
  • proposition thmcounterproposition
  • ...and 6 more