Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Lipschitz Bandits

Bongsoo Yi, Yue Kang, Yao Li

TL;DR

This work tackles Lipschitz bandits with continuous arm spaces by introducing the first quantum algorithms, Q-LAE and Q-Zooming, that leverage quantum Monte Carlo to accelerate mean estimation over adaptive discretizations. By rethinking the zooming-dimension framework and employing a quantum elimination or stage-based zooming strategy, the authors achieve regret bounds of order O(T^{d_z/(d_z+1)}) under standard noise models, improving upon classical bounds that scale as roughly O(T^{(d_z+1)/(d_z+2)}). Empirical results on representative Lipschitz problems show superior performance of the quantum methods over classical Zooming, with Q-LAE often offering robust practical performance due to its aggressive elimination of suboptimal regions. The paper demonstrates the potential of quantum techniques to enhance learning in continuous-action bandits, offering new directions for combining quantum subroutines with structured exploration strategies in online learning.

Abstract

The Lipschitz bandit is a key variant of stochastic bandit problems where the expected reward function satisfies a Lipschitz condition with respect to an arm metric space. With its wide-ranging practical applications, various Lipschitz bandit algorithms have been developed, achieving the cumulative regret lower bound of order $\tilde O(T^{(d_z+1)/(d_z+2)})$ over time horizon $T$. Motivated by recent advancements in quantum computing and the demonstrated success of quantum Monte Carlo in simpler bandit settings, we introduce the first quantum Lipschitz bandit algorithms to address the challenges of continuous action spaces and non-linear reward functions. Specifically, we first leverage the elimination-based framework to propose an efficient quantum Lipschitz bandit algorithm named Q-LAE. Next, we present novel modifications to the classical Zooming algorithm, which results in a simple quantum Lipschitz bandit method, Q-Zooming. Both algorithms exploit the computational power of quantum methods to achieve an improved regret bound of $\tilde O(T^{d_z/(d_z+1)})$. Comprehensive experiments further validate our improved theoretical findings, demonstrating superior empirical performance compared to existing Lipschitz bandit methods.

Quantum Lipschitz Bandits

TL;DR

This work tackles Lipschitz bandits with continuous arm spaces by introducing the first quantum algorithms, Q-LAE and Q-Zooming, that leverage quantum Monte Carlo to accelerate mean estimation over adaptive discretizations. By rethinking the zooming-dimension framework and employing a quantum elimination or stage-based zooming strategy, the authors achieve regret bounds of order O(T^{d_z/(d_z+1)}) under standard noise models, improving upon classical bounds that scale as roughly O(T^{(d_z+1)/(d_z+2)}). Empirical results on representative Lipschitz problems show superior performance of the quantum methods over classical Zooming, with Q-LAE often offering robust practical performance due to its aggressive elimination of suboptimal regions. The paper demonstrates the potential of quantum techniques to enhance learning in continuous-action bandits, offering new directions for combining quantum subroutines with structured exploration strategies in online learning.

Abstract

The Lipschitz bandit is a key variant of stochastic bandit problems where the expected reward function satisfies a Lipschitz condition with respect to an arm metric space. With its wide-ranging practical applications, various Lipschitz bandit algorithms have been developed, achieving the cumulative regret lower bound of order over time horizon . Motivated by recent advancements in quantum computing and the demonstrated success of quantum Monte Carlo in simpler bandit settings, we introduce the first quantum Lipschitz bandit algorithms to address the challenges of continuous action spaces and non-linear reward functions. Specifically, we first leverage the elimination-based framework to propose an efficient quantum Lipschitz bandit algorithm named Q-LAE. Next, we present novel modifications to the classical Zooming algorithm, which results in a simple quantum Lipschitz bandit method, Q-Zooming. Both algorithms exploit the computational power of quantum methods to achieve an improved regret bound of . Comprehensive experiments further validate our improved theoretical findings, demonstrating superior empirical performance compared to existing Lipschitz bandit methods.

Paper Structure

This paper contains 26 sections, 13 theorems, 71 equations, 1 figure, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setting and Preliminaries
  4. Lipschitz Bandits
  5. Zooming Dimension.
  6. Covering and Packing.
  7. Quantum Computation
  8. Basics of Quantum States and Measurements.
  9. Quantum Reward Oracle and Bandit Settings.
  10. Quantum Monte Carlo (QMC) Method.
  11. Quantum Lipschitz Adaptive Elimination Algorithm
  12. Regret Analysis
  13. Quantum Zooming Algorithm
  14. Regret Analysis
  15. Experiments
  16. ...and 11 more sections

Key Result

Lemma 3.4

Let $y : \Omega \to \mathbb{R}$ be a random variable with bounded variance, where $\Omega$ is equipped with a probability measure $P$, and the quantum unitary oracle $\mathcal{O}$ encodes $P$ and $y$. With certain assumption about the noise, the QMC method offers the following guarantees:

Figures (1)

  • Figure 1: Average Cumulative Regret. The regret performance for each function and noise model is evaluated as the average cumulative regret over 30 independent runs, with the corresponding standard deviations also reported.

Theorems & Definitions (31)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Lemma 3.4: Quantum Monte Carlo method montanaro2015quantum
  • Remark 4.1
  • Theorem 4.2
  • Theorem 4.3
  • Theorem 5.1
  • Remark 5.2
  • Theorem 5.3
  • ...and 21 more