Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tighter Confidence Bounds for Sequential Kernel Regression

Hamish Flynn, David Reeb

TL;DR

This work uses martingale tail inequalities to establish new confidence bounds for sequential kernel regression, and it is proved that their new confidence bounds are always tighter than existing ones in this setting.

Abstract

Confidence bounds are an essential tool for rigorously quantifying the uncertainty of predictions. They are a core component in many sequential learning and decision-making algorithms, with tighter confidence bounds giving rise to algorithms with better empirical performance and better performance guarantees. In this work, we use martingale tail inequalities to establish new confidence bounds for sequential kernel regression. Our confidence bounds can be computed by solving a conic program, although this bare version quickly becomes impractical, because the number of variables grows with the sample size. However, we show that the dual of this conic program allows us to efficiently compute tight confidence bounds. We prove that our new confidence bounds are always tighter than existing ones in this setting. We apply our confidence bounds to kernel bandit problems, and we find that when our confidence bounds replace existing ones, the KernelUCB (GP-UCB) algorithm has better empirical performance, a matching worst-case performance guarantee and comparable computational cost.

Tighter Confidence Bounds for Sequential Kernel Regression

TL;DR

This work uses martingale tail inequalities to establish new confidence bounds for sequential kernel regression, and it is proved that their new confidence bounds are always tighter than existing ones in this setting.

Abstract

Confidence bounds are an essential tool for rigorously quantifying the uncertainty of predictions. They are a core component in many sequential learning and decision-making algorithms, with tighter confidence bounds giving rise to algorithms with better empirical performance and better performance guarantees. In this work, we use martingale tail inequalities to establish new confidence bounds for sequential kernel regression. Our confidence bounds can be computed by solving a conic program, although this bare version quickly becomes impractical, because the number of variables grows with the sample size. However, we show that the dual of this conic program allows us to efficiently compute tight confidence bounds. We prove that our new confidence bounds are always tighter than existing ones in this setting. We apply our confidence bounds to kernel bandit problems, and we find that when our confidence bounds replace existing ones, the KernelUCB (GP-UCB) algorithm has better empirical performance, a matching worst-case performance guarantee and comparable computational cost.
Paper Structure (47 sections, 29 theorems, 175 equations, 7 figures, 1 table, 1 algorithm)

This paper contains 47 sections, 29 theorems, 175 equations, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM STATEMENT
  3. Notation.
  4. RELATED WORK
  5. CONFIDENCE BOUNDS FOR KERNEL REGRESSION
  6. Martingale Mixture Tail Bounds
  7. Confidence Sequences
  8. Implicit Confidence Bounds
  9. Explicit Confidence Bounds
  10. Confidence Bound Maximisation
  11. KERNEL BANDITS
  12. Stochastic Kernel Bandits
  13. Kernel Bandit Algorithms
  14. Kernel CMM-UCB
  15. Kernel DMM-UCB
  16. ...and 32 more sections

Key Result

Theorem 4.1

For any $\delta \in (0, 1)$, any sequence of distributions $(P_t| t \in \mathbb{N})$ satisfying (a) and (b), and any sequence of predictable random variables $(\lambda_t| t \in \mathbb{N})$, it holds with probability at least $1 - \delta$ that

Figures (7)

  • Figure 1: The upper and lower confidence bounds used by our Kernel CMM-UCB method (left), the confidence bounds from abbasi2012online (AY-GP-UCB) (middle-right), and the confidence bounds used by Improved GP-UCB (IGP-UCB) chowdhury2017kernelized (right) for a Matérn kernel test function with smoothness $\nu = 3/2$ and lengthscale $\ell = 0.5$. Our Kernel CMM-UCB method -- and its relaxed versions DMM-UCB and AMM-UCB -- produces confidence bounds that are visibly closer to the ground-truth function (dashed line) than those of AY-GP-UCB and IGP-UCB (cf. also Sec. \ref{['sec:tighter_conf']}).
  • Figure 2: Cumulative regret of our KernelUCB-style algorithms as well as AY-GP-UCB, IGP-UCB and a random baseline over $T=1000$ rounds for $d=3$ and various kernels (columns) and length scales (rows). We show the mean $\pm$ standard deviation over 10 repetitions.
  • Figure 3: Python code for solving (\ref{['eqn:ucb_cone_prog']}) with CVXPY.
  • Figure 4: Wall-clock time per step for our kernel UCB-style algorithms as well as AY-GP-UCB, IGP-UCB over $T=1000$ rounds with: (left) original $y$-axis; (middle) zoomed-in $y$-axis; (right) log scale $y$-axis. We show the mean over 10 repetitions.
  • Figure 5: Cumulative regret of our KernelUCB-style algorithms as well as AY-GP-UCB, IGP-UCB and a random baseline over $T=1000$ rounds for $d=2$ and various kernels (columns) and length scales (rows). We show the mean $\pm$ standard deviation over 10 repetitions.
  • ...and 2 more figures

Theorems & Definitions (49)

  • Theorem 4.1: Theorem 5.1 of flynn2023improved
  • Lemma 4.2
  • Corollary 4.3
  • Theorem 4.4
  • Theorem 4.5
  • Corollary 4.6
  • Theorem 6.1
  • Theorem 6.2
  • Lemma A.1: flynn2023improved
  • Lemma B.1
  • ...and 39 more