Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Oracle-Efficient Combinatorial Semi-Bandits

Jung-hun Kim, Milan Vojnović, Min-hwan Oh

TL;DR

The paper addresses the scalability challenge of combinatorial semi-bandits by proposing two oracle-efficient frameworks—adaptive rare oracle queries (AROQ) and scheduled rare oracle queries (SROQ)—that dramatically reduce oracle adaptivity and query complexity while preserving tight gap-free regret in worst-case linear, covariance-dependent, and general-reward settings. It introduces covariance-adaptive variants and extends to non-linear rewards, achieving near-optimal regret with logarithmic or doubly-logarithmic oracle costs. Theoretical results are complemented by experiments showing substantial reductions in oracle usage and improved runtime, supporting practical deployment in large action spaces. Overall, the work significantly enhances the practicality of many real-world applications of combinatorial bandits by alleviating the computational burden of frequent oracle calls.

Abstract

We study the combinatorial semi-bandit problem where an agent selects a subset of base arms and receives individual feedback. While this generalizes the classical multi-armed bandit and has broad applicability, its scalability is limited by the high cost of combinatorial optimization, requiring oracle queries at every round. To tackle this, we propose oracle-efficient frameworks that significantly reduce oracle calls while maintaining tight regret guarantees. For the worst-case linear reward setting, our algorithms achieve $\tilde{O}(\sqrt{T})$ regret using only $O(\log\log T)$ oracle queries. We also propose covariance-adaptive algorithms that leverage noise structure for improved regret, and extend our approach to general (non-linear) rewards. Overall, our methods reduce oracle usage from linear to (doubly) logarithmic in time, with strong theoretical guarantees.

Oracle-Efficient Combinatorial Semi-Bandits

TL;DR

The paper addresses the scalability challenge of combinatorial semi-bandits by proposing two oracle-efficient frameworks—adaptive rare oracle queries (AROQ) and scheduled rare oracle queries (SROQ)—that dramatically reduce oracle adaptivity and query complexity while preserving tight gap-free regret in worst-case linear, covariance-dependent, and general-reward settings. It introduces covariance-adaptive variants and extends to non-linear rewards, achieving near-optimal regret with logarithmic or doubly-logarithmic oracle costs. Theoretical results are complemented by experiments showing substantial reductions in oracle usage and improved runtime, supporting practical deployment in large action spaces. Overall, the work significantly enhances the practicality of many real-world applications of combinatorial bandits by alleviating the computational burden of frequent oracle calls.

Abstract

We study the combinatorial semi-bandit problem where an agent selects a subset of base arms and receives individual feedback. While this generalizes the classical multi-armed bandit and has broad applicability, its scalability is limited by the high cost of combinatorial optimization, requiring oracle queries at every round. To tackle this, we propose oracle-efficient frameworks that significantly reduce oracle calls while maintaining tight regret guarantees. For the worst-case linear reward setting, our algorithms achieve regret using only oracle queries. We also propose covariance-adaptive algorithms that leverage noise structure for improved regret, and extend our approach to general (non-linear) rewards. Overall, our methods reduce oracle usage from linear to (doubly) logarithmic in time, with strong theoretical guarantees.
Paper Structure (46 sections, 19 theorems, 65 equations, 4 figures, 1 table)

This paper contains 46 sections, 19 theorems, 65 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work.
  3. Problem Formulation
  4. Regret.
  5. Combinatorial Optimization.
  6. Oracle Efficiency.
  7. Additional Notation.
  8. Oracle-Efficient Algorithms and Regret Analysis
  9. Adaptive Rare Oracle Queries
  10. Comparison to Previous Work.
  11. $\alpha$-Approximation Oracle.
  12. Scheduled Rare Oracle Queries
  13. Extension to Covariance-dependent CMAB
  14. Adaptive Rare Oracle Queries for Covariance-adaptive Approach
  15. Comparison to Previous Work.
  16. ...and 31 more sections

Key Result

Theorem 1

With oracle adaptivity and query complexities of $O(d\log \log (Tm/d))$, respectively, Algorithm alg:adaptive achieves a regret bound of

Figures (4)

  • Figure 1: Oracle adaptivity and query complexities.
  • Figure 2: Experimental results for linear rewards with $d=20$ and $m=3$.
  • Figure 3: Experimental results for covariance-adaptivity under linear reward with $d=10$ and $m=3$: (a) cumulative oracle adaptivity complexity, (b) cumulative oracle query complexity, (c) regret, (d) runtime, (e) overall oracle adaptivity complexity, and (f) overall oracle query complexity of algorithms.
  • Figure 4: Experimental results for non-linear reward with $d=5$ and $m=2$: (a) cumulative oracle adaptivity complexity, (b) cumulative oracle query complexity, (c) regret, (d) runtime, (e) overall oracle adaptivity complexity, and (f) overall oracle query complexity of algorithms.

Theorems & Definitions (20)

  • Theorem 1
  • Theorem 2
  • Remark 1
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Theorem 6
  • Lemma 1: Oracle Queries Bound
  • Lemma 3: Azuma-Hoeffding Inequality
  • Theorem 7
  • ...and 10 more