Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lower Bounds for Time-Varying Kernelized Bandits

Xu Cai, Jonathan Scarlett

TL;DR

This work addresses non-stationary kernelized bandits by quantifying fundamental limits when the objective function sequence ${f_t}$ evolves with a total variation budget under either ${\ell_{\infty}}$ or RKHS norms. By extending stationary bump-based lower bounds to a time-varying setting and partitioning the horizon into blocks, the authors derive three main algorithm-independent lower bounds that depend on the time horizon ${T}$, variation budget ${\Delta}$ (or ${L}$ switches), and Matérn kernel parameters ${\nu,d}$. The results are then contrasted with contemporary upper bounds (notably from Hon23) to reveal near-tight scaling in several regimes, while highlighting notable gaps in RKHS-norm variation that motivate further work. The work also discusses concurrent results that tighten the ${\ell_{\infty}}$ case and outlines open problems about potential regime-specific differences between ${\dagger=\infty}$ and ${\dagger=k}$ variations and the tightness of certain lower bounds for finite smoothness ${\nu,d}$.

Abstract

The optimization of black-box functions with noisy observations is a fundamental problem with widespread applications, and has been widely studied under the assumption that the function lies in a reproducing kernel Hilbert space (RKHS). This problem has been studied extensively in the stationary setting, and near-optimal regret bounds are known via developments in both upper and lower bounds. In this paper, we consider non-stationary scenarios, which are crucial for certain applications but are currently less well-understood. Specifically, we provide the first algorithm-independent lower bounds, where the time variations are subject satisfying a total variation budget according to some function norm. Under $\ell_{\infty}$-norm variations, our bounds are found to be close to an existing upper bound (Hong et al., 2023). Under RKHS norm variations, the upper and lower bounds are still reasonably close but with more of a gap, raising the interesting open question of whether non-minor improvements in the upper bound are possible.

Lower Bounds for Time-Varying Kernelized Bandits

TL;DR

This work addresses non-stationary kernelized bandits by quantifying fundamental limits when the objective function sequence evolves with a total variation budget under either or RKHS norms. By extending stationary bump-based lower bounds to a time-varying setting and partitioning the horizon into blocks, the authors derive three main algorithm-independent lower bounds that depend on the time horizon , variation budget (or switches), and Matérn kernel parameters . The results are then contrasted with contemporary upper bounds (notably from Hon23) to reveal near-tight scaling in several regimes, while highlighting notable gaps in RKHS-norm variation that motivate further work. The work also discusses concurrent results that tighten the case and outlines open problems about potential regime-specific differences between and variations and the tightness of certain lower bounds for finite smoothness .

Abstract

The optimization of black-box functions with noisy observations is a fundamental problem with widespread applications, and has been widely studied under the assumption that the function lies in a reproducing kernel Hilbert space (RKHS). This problem has been studied extensively in the stationary setting, and near-optimal regret bounds are known via developments in both upper and lower bounds. In this paper, we consider non-stationary scenarios, which are crucial for certain applications but are currently less well-understood. Specifically, we provide the first algorithm-independent lower bounds, where the time variations are subject satisfying a total variation budget according to some function norm. Under -norm variations, our bounds are found to be close to an existing upper bound (Hong et al., 2023). Under RKHS norm variations, the upper and lower bounds are still reasonably close but with more of a gap, raising the interesting open question of whether non-minor improvements in the upper bound are possible.

Paper Structure

This paper contains 25 sections, 7 theorems, 36 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup
  4. Main Results
  5. Theorem Statements
  6. Comparisons to Upper Bounds
  7. Outline of Proofs
  8. Open Problems
  9. Conclusion
  10. Note on Notation
  11. Tools for Proving the Lower Bounds
  12. Stationary Setting
  13. Non-Stationary Setting
  14. Proofs of Main Results
  15. Proof of Theorem \ref{['thm:L']} (At Most $L-1$ Changes)
  16. ...and 10 more sections

Key Result

Theorem 1

(Limited Number of Changes) Consider the setup in which the function changes at most $L-1$ times according to eq:L_changes, and suppose that the RKHS norm $B$, noise level $\sigma$, dimension $d$, and Matérn parameters $(\nu,\ell)$ are all constant. Then, with time horizon $T$ and $L \in \{1,\dotsc, Moreover, $\mathbb{E}[R_T] \ge \Omega( \sqrt{LT} )$ for all $(\nu,d)$.

Figures (6)

  • Figure 1: Comparison of upper and lower bounds when $\Delta = \Theta(1)$. (Top) Values of $\alpha$ such that the regret has dependence $T^{\alpha}$. (Bottom) Differences of the upper bound's $\alpha$ value and the lower bound's value. For $\ell_{\infty}$-norm, the highest difference is below 0.035.
  • Figure 2: Comparison of upper and lower bounds when $\Delta = \Theta(T^{0.1})$. (Top) Value $\alpha$ such that the regret has dependence $T^{\alpha}$. (Bottom) Differences of the upper bound's $\alpha$ value and the lower bound's value. For $\ell_{\infty}$-norm, the highest difference is around 0.05.
  • Figure 3: Comparison of upper and lower bounds when $\Delta = \Theta(\sqrt{T})$. (Top) Value $\alpha$ such that the regret has dependence $T^{\alpha}$. (Bottom) Differences of the upper bound's $\alpha$ value and the lower bound's value. For $\ell_{\infty}$-norm, the highest difference is below 0.1.
  • Figure 4: Comparison of upper and lower bounds when $\Delta = \Theta(T^{0.9})$. (Top) Value $\alpha$ such that the regret has dependence $T^{\alpha}$. (Bottom) Differences of the upper bound's $\alpha$ value and the lower bound's value. For $\ell_{\infty}$-norm, the highest difference is around 0.03.
  • Figure 5: Comparison of lower bounds for $\ell_{\infty}$-norm variation and RKHS norm variation for various $\beta$ values such that $\Delta = \Theta(T^{\beta})$. Specifically, the difference is between the two values of $\alpha$ for which the regret has dependence $T^{\alpha}$.
  • ...and 1 more figures

Theorems & Definitions (7)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Lemma 1
  • Lemma 2
  • Theorem 4
  • Lemma 3