Quantum-Enhanced Neural Contextual Bandit Algorithms
Yuqi Huang, Vincent Y. F Tan, Sharu Theresa Jose
TL;DR
This paper tackles the challenge of online contextual bandits with non-linear rewards by leveraging quantum neural networks without training them, using the Quantum Neural Tangent Kernel (QNTK) as a static kernel. The proposed QNTK-UCB algorithm freezes the QNN at random initialization and performs ridge regression in the QNTK feature space, achieving a regret bound of $\tilde{O}(\tilde{d}_{\text{q}}\sqrt{T})$ where $\tilde{d}_{\text{q}}$ is the quantum effective dimension. Theoretical results show a dramatic reduction in parameter requirements compared with classical neural bandits, and experiments on non-linear synthetic benchmarks and VQE-related tasks demonstrate improved sample efficiency in low-data regimes due to the quantum inductive bias. The work highlights how concentration and spectral properties of the QNTK can provide implicit regularization and enable quantum advantage in online learning, while also outlining future directions for hybrid quantum-classical models to balance expressivity and trainability.
Abstract
Stochastic contextual bandits are fundamental for sequential decision-making but pose significant challenges for existing neural network-based algorithms, particularly when scaling to quantum neural networks (QNNs) due to issues such as massive over-parameterization, computational instability, and the barren plateau phenomenon. This paper introduces the Quantum Neural Tangent Kernel-Upper Confidence Bound (QNTK-UCB) algorithm, a novel algorithm that leverages the Quantum Neural Tangent Kernel (QNTK) to address these limitations. By freezing the QNN at a random initialization and utilizing its static QNTK as a kernel for ridge regression, QNTK-UCB bypasses the unstable training dynamics inherent in explicit parameterized quantum circuit training while fully exploiting the unique quantum inductive bias. For a time horizon $T$ and $K$ actions, our theoretical analysis reveals a significantly improved parameter scaling of $Ω((TK)^3)$ for QNTK-UCB, a substantial reduction compared to $Ω((TK)^8)$ required by classical NeuralUCB algorithms for similar regret guarantees. Empirical evaluations on non-linear synthetic benchmarks and quantum-native variational quantum eigensolver tasks demonstrate QNTK-UCB's superior sample efficiency in low-data regimes. This work highlights how the inherent properties of QNTK provide implicit regularization and a sharper spectral decay, paving the way for achieving ``quantum advantage'' in online learning.