Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Linear Bandits with Limited Adaptivity

Ayush Sawarni, Nirjhar Das, Siddharth Barman, Gaurav Sinha

TL;DR

This work tackles generalized linear contextual bandits under limited adaptivity by introducing two algorithms, B-GLinCB for the batched (M1) setting and RS-GLinCB for the rarely-switching (M2) setting. Both algorithms achieve near-optimal regret $\tilde{O}(\sqrt{T})$ while removing dependence on the non-linearity parameter $\kappa$ in the leading term, leveraging GLM self-concordance and distributional optimal design to extend beyond linear or logistic models. B-GLinCB uses upfront batch design and scaled-arm distributions to maintain informative designs across batches, while RS-GLinCB employs context-dependent switching criteria to bound policy updates and preserve computational efficiency. Together, the results enable practical, scalable GLM contextual bandits under limited adaptivity, with theoretical guarantees and empirical validation across logistic and probit settings.

Abstract

We study the generalized linear contextual bandit problem within the constraints of limited adaptivity. In this paper, we present two algorithms, $\texttt{B-GLinCB}$ and $\texttt{RS-GLinCB}$, that address, respectively, two prevalent limited adaptivity settings. Given a budget $M$ on the number of policy updates, in the first setting, the algorithm needs to decide upfront $M$ rounds at which it will update its policy, while in the second setting it can adaptively perform $M$ policy updates during its course. For the first setting, we design an algorithm $\texttt{B-GLinCB}$, that incurs $\tilde{O}(\sqrt{T})$ regret when $M = Ω( \log{\log T} )$ and the arm feature vectors are generated stochastically. For the second setting, we design an algorithm $\texttt{RS-GLinCB}$ that updates its policy $\tilde{O}(\log^2 T)$ times and achieves a regret of $\tilde{O}(\sqrt{T})$ even when the arm feature vectors are adversarially generated. Notably, in these bounds, we manage to eliminate the dependence on a key instance dependent parameter $κ$, that captures non-linearity of the underlying reward model. Our novel approach for removing this dependence for generalized linear contextual bandits might be of independent interest.

Generalized Linear Bandits with Limited Adaptivity

TL;DR

This work tackles generalized linear contextual bandits under limited adaptivity by introducing two algorithms, B-GLinCB for the batched (M1) setting and RS-GLinCB for the rarely-switching (M2) setting. Both algorithms achieve near-optimal regret while removing dependence on the non-linearity parameter in the leading term, leveraging GLM self-concordance and distributional optimal design to extend beyond linear or logistic models. B-GLinCB uses upfront batch design and scaled-arm distributions to maintain informative designs across batches, while RS-GLinCB employs context-dependent switching criteria to bound policy updates and preserve computational efficiency. Together, the results enable practical, scalable GLM contextual bandits under limited adaptivity, with theoretical guarantees and empirical validation across logistic and probit settings.

Abstract

We study the generalized linear contextual bandit problem within the constraints of limited adaptivity. In this paper, we present two algorithms, and , that address, respectively, two prevalent limited adaptivity settings. Given a budget on the number of policy updates, in the first setting, the algorithm needs to decide upfront rounds at which it will update its policy, while in the second setting it can adaptively perform policy updates during its course. For the first setting, we design an algorithm , that incurs regret when and the arm feature vectors are generated stochastically. For the second setting, we design an algorithm that updates its policy times and achieves a regret of even when the arm feature vectors are adversarially generated. Notably, in these bounds, we manage to eliminate the dependence on a key instance dependent parameter , that captures non-linearity of the underlying reward model. Our novel approach for removing this dependence for generalized linear contextual bandits might be of independent interest.
Paper Structure (27 sections, 44 theorems, 129 equations, 1 figure, 2 algorithms)

This paper contains 27 sections, 44 theorems, 129 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Important Remarks on Contributions and Comparison with Prior Work
  4. Notations and Preliminaries
  5. Instance Dependent Non-Linearity Parameters
  6. Optimal Design Policies
  7. B-GLinCB
  8. RS-GLinCB
  9. Experiments
  10. Conclusion and Future Work
  11. Regret Analysis of B-GLinCB
  12. Additional Notation
  13. Concentration Inequalities and Confidence Intervals
  14. Preliminary Lemmas
  15. Proof of Theorem \ref{['thm:batch_algo_regret']}
  16. ...and 12 more sections

Key Result

Lemma 2.2

For any GLM supported on $[0,R]$ almost surely, the link function $\mu(\cdot)$ satisfies $\lvert \ddot{\mu}(z) \rvert \leq R \dot{\mu}(z)$, for all $z \in \mathbb{R}$.

Figures (1)

  • Figure 1: Top: Cumulative Regret vs. number of rounds for Logistic (left) and Probit (right) reward models. Bottom: (left) Execution times of ECOLog and RS-GLinCB for different values of $\kappa$ (low $\kappa = 9.3$ and high $\kappa = 141.6$) for Logistic rewards. (right) Execution times of GLOC and RS-GLinCB for different values of $\kappa$ (low $\kappa = 17.6$ and high $\kappa = 202.3$) for Probit rewards.

Theorems & Definitions (89)

  • Remark 1.1: $\kappa$-independence
  • Remark 1.2: Computational efficiency
  • Remark 1.3: Self Concordance of bounded GLMs
  • Lemma 2.2: Self-Concordance of GLMs
  • Remark 2.3
  • Lemma 2.5: Keifer-Wolfowitz
  • Lemma 2.5: Theorem 5, ruan2021linear
  • Remark 3.1
  • Theorem 3.2
  • Corollary 3.3
  • ...and 79 more