Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Logistic Bandits

Seoungbin Bae, Dabeen Lee

TL;DR

This work addresses neural logistic bandits where rewards follow a logistic link and are approximated by deep nets. It introduces a Bernstein‑type tail inequality for self‑normalized vector martingales that is variance‑ and data‑adaptive, enabling regret that scales with an effective dimension $\widetilde{d}$ rather than the full feature dimension. Two algorithms, NeuralLog-UCB-1 and NeuralLog-UCB-2, achieve improved regret bounds $\widetilde{O}(\widetilde{d}\sqrt{\kappa T})$ and $\widetilde{O}(\widetilde{d}\sqrt{T/\kappa^{*}})$, with the latter using a neural network–estimated variance to avoid the worst‑case variance, and empirical results validate the theory on synthetic and real data. The work advances neural contextual bandits by delivering variance‑adaptive, data‑driven regret guarantees and practical UCB schemes that mitigate dependence on both ambient dimension and the variance parameter $\kappa$.

Abstract

We study the problem of neural logistic bandits, where the main task is to learn an unknown reward function within a logistic link function using a neural network. Existing approaches either exhibit unfavorable dependencies on $κ$, where $1/κ$ represents the minimum variance of reward distributions, or suffer from direct dependence on the feature dimension $d$, which can be huge in neural network-based settings. In this work, we introduce a novel Bernstein-type inequality for self-normalized vector-valued martingales that is designed to bypass a direct dependence on the ambient dimension. This lets us deduce a regret upper bound that grows with the effective dimension $\widetilde{d}$, not the feature dimension, while keeping a minimal dependence on $κ$. Based on the concentration inequality, we propose two algorithms, NeuralLog-UCB-1 and NeuralLog-UCB-2, that guarantee regret upper bounds of order $\widetilde{O}(\widetilde{d}\sqrt{κT})$ and $\widetilde{O}(\widetilde{d}\sqrt{T/κ})$, respectively, improving on the existing results. Lastly, we report numerical results on both synthetic and real datasets to validate our theoretical findings.

Neural Logistic Bandits

TL;DR

This work addresses neural logistic bandits where rewards follow a logistic link and are approximated by deep nets. It introduces a Bernstein‑type tail inequality for self‑normalized vector martingales that is variance‑ and data‑adaptive, enabling regret that scales with an effective dimension rather than the full feature dimension. Two algorithms, NeuralLog-UCB-1 and NeuralLog-UCB-2, achieve improved regret bounds and , with the latter using a neural network–estimated variance to avoid the worst‑case variance, and empirical results validate the theory on synthetic and real data. The work advances neural contextual bandits by delivering variance‑adaptive, data‑driven regret guarantees and practical UCB schemes that mitigate dependence on both ambient dimension and the variance parameter .

Abstract

We study the problem of neural logistic bandits, where the main task is to learn an unknown reward function within a logistic link function using a neural network. Existing approaches either exhibit unfavorable dependencies on , where represents the minimum variance of reward distributions, or suffer from direct dependence on the feature dimension , which can be huge in neural network-based settings. In this work, we introduce a novel Bernstein-type inequality for self-normalized vector-valued martingales that is designed to bypass a direct dependence on the ambient dimension. This lets us deduce a regret upper bound that grows with the effective dimension , not the feature dimension, while keeping a minimal dependence on . Based on the concentration inequality, we propose two algorithms, NeuralLog-UCB-1 and NeuralLog-UCB-2, that guarantee regret upper bounds of order and , respectively, improving on the existing results. Lastly, we report numerical results on both synthetic and real datasets to validate our theoretical findings.
Paper Structure (29 sections, 23 theorems, 153 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 29 sections, 23 theorems, 153 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Variance- and data-adaptive self-normalized martingale tail inequality
  4. Neural logistic bandits with improved UCB
  5. Main results
  6. Regret analysis
  7. Refined algorithm with neural network-estimated variance
  8. Experiments
  9. Conclusion and future work
  10. Deferred experiments from \ref{['sec:exp']}
  11. Related works
  12. Useful lemmas for neural bandits
  13. Proof of \ref{['lem:close']}
  14. Proof of \ref{['thm:main']}
  15. Proof of \ref{['lem:b.3']}
  16. ...and 14 more sections

Key Result

Theorem 3.1

Let $\{\mathcal{G}_t\}_{t=1}^\infty$ be a filtration, and $\{x_t, \eta_t\}_{t\geq 1}$ be a stochastic process where $x_t \in \mathbb{R}^d$ is $\mathcal{G}_{t}$-measurable and $\eta_t \in \mathbb{R}$ is $\mathcal{G}_{t+1}$-measurable. Suppose there exist constants $M,R,L, \lambda> 0$ and the paramete Then, for any $0< \delta < 1$ and any $t>0$, with probability at least $1-\delta$:

Figures (3)

  • Figure 1: Comparison of cumulative regret of baseline algorithms for nonlinear reward functions.
  • Figure 2: Comparison of cumulative regret of baseline algorithms for real-world dataset.
  • Figure 3: Comparison of cumulative regret of baseline algorithms with varying effective dimension $\widetilde{d}$.

Theorems & Definitions (33)

  • Theorem 3.1
  • Definition 5.3
  • Theorem 5.5
  • Remark 5.6
  • Lemma 6.1: Lemma 5.1, zhou2020neural
  • Definition 6.2
  • Lemma 6.4
  • Lemma 6.5
  • proof : Proof sketch of \ref{['thm:regret']}
  • Lemma 7.1
  • ...and 23 more