Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Combinatorial Multi-armed Bandits with Probabilistically Triggered Arms

Kongchang Zhou, Tingyu Zhang, Wei Chen, Fang Kong

TL;DR

The paper addresses learning in combinatorial multi-armed bandits with probabilistically triggered arms by integrating offline data with online interactions. It introduces the hybrid CMAB-T framework and the hybrid CUCB algorithm, which uses two confidence bounds—one online and one bias-corrected hybrid bound—to adaptively leverage offline data through a bias vector $V$. Theoretical results establish gap-dependent and gap-independent regret guarantees that quantify how informative offline data reduces exploration costs via a data-dependent term $N_i'$, while recovering purely online performance when offline data is unhelpful. Empirical results on synthetic CMAB-T problems and real-world data (MovieLens) demonstrate robust improvements over both purely online and offline baselines, including under distributional bias.

Abstract

The problem of combinatorial multi-armed bandits with probabilistically triggered arms (CMAB-T) has been extensively studied. Prior work primarily focuses on either the online setting where an agent learns about the unknown environment through iterative interactions, or the offline setting where a policy is learned solely from logged data. However, each of these paradigms has inherent limitations: online algorithms suffer from high interaction costs and slow adaptation, while offline methods are constrained by dataset quality and lack of exploration capabilities. To address these complementary weaknesses, we propose hybrid CMAB-T, a new framework that integrates offline data with online interaction in a principled manner. Our proposed hybrid CUCB algorithm leverages offline data to guide exploration and accelerate convergence, while strategically incorporating online interactions to mitigate the insufficient coverage or distributional bias of the offline dataset. We provide theoretical guarantees on the algorithm's regret, demonstrating that hybrid CUCB significantly outperforms purely online approaches when high-quality offline data is available, and effectively corrects the bias inherent in offline-only methods when the data is limited or misaligned. Empirical results further demonstrate the consistent advantage of our algorithm.

Hybrid Combinatorial Multi-armed Bandits with Probabilistically Triggered Arms

TL;DR

The paper addresses learning in combinatorial multi-armed bandits with probabilistically triggered arms by integrating offline data with online interactions. It introduces the hybrid CMAB-T framework and the hybrid CUCB algorithm, which uses two confidence bounds—one online and one bias-corrected hybrid bound—to adaptively leverage offline data through a bias vector . Theoretical results establish gap-dependent and gap-independent regret guarantees that quantify how informative offline data reduces exploration costs via a data-dependent term , while recovering purely online performance when offline data is unhelpful. Empirical results on synthetic CMAB-T problems and real-world data (MovieLens) demonstrate robust improvements over both purely online and offline baselines, including under distributional bias.

Abstract

The problem of combinatorial multi-armed bandits with probabilistically triggered arms (CMAB-T) has been extensively studied. Prior work primarily focuses on either the online setting where an agent learns about the unknown environment through iterative interactions, or the offline setting where a policy is learned solely from logged data. However, each of these paradigms has inherent limitations: online algorithms suffer from high interaction costs and slow adaptation, while offline methods are constrained by dataset quality and lack of exploration capabilities. To address these complementary weaknesses, we propose hybrid CMAB-T, a new framework that integrates offline data with online interaction in a principled manner. Our proposed hybrid CUCB algorithm leverages offline data to guide exploration and accelerate convergence, while strategically incorporating online interactions to mitigate the insufficient coverage or distributional bias of the offline dataset. We provide theoretical guarantees on the algorithm's regret, demonstrating that hybrid CUCB significantly outperforms purely online approaches when high-quality offline data is available, and effectively corrects the bias inherent in offline-only methods when the data is limited or misaligned. Empirical results further demonstrate the consistent advantage of our algorithm.
Paper Structure (20 sections, 5 theorems, 69 equations, 3 figures, 1 algorithm)

This paper contains 20 sections, 5 theorems, 69 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup
  4. Offline --approximation oracle $\mathcal{O}$.
  5. The Hybrid CUCB Algorithm
  6. Theoretical Analysis
  7. Gap-Dependent Bound
  8. Gap-Independent Bound
  9. Experiments
  10. Conclusion
  11. Technical Lemmas
  12. Proof of Theorem \ref{['thm:regret-bound']}
  13. Case 1
  14. Case 2
  15. Proof of Theorem \ref{['thm:gap-indep']}
  16. ...and 5 more sections

Key Result

Theorem 1

For an H-CMAB-T problem $([m], \mathcal{S}, \mathcal{D}, D^{\text{trig}}, R, \mathcal{B})$ that satisfies monotonicity (Condition 1) and TPM bounded smoothness (Condition 2), the hybrid CUCB algorithm with an input bias control vector $V$ and an $(\alpha, \beta)$-approximation oracle achieves an $(\ where

Figures (3)

  • Figure 1: Performance comparison of hybrid CUCB against baselines with unbiased offline data set.
  • Figure 2: Performance comparison of hybrid CUCB against baselines with the biased offline data set.
  • Figure 3: Performance comparison of hybrid CUCB against baselines in a real-world dataset.

Theorems & Definitions (13)

  • Definition 1: --approximation regret.
  • Definition 2: Gap wang2017cmabt
  • Theorem 1: Gap-Dependent Regret Bound
  • Theorem 2: Gap-Independent Regret Bound
  • Definition 3: Event-Filtered Regret wang2017cmabt
  • Lemma 1
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • ...and 3 more