Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles

Jung-hun Kim, Se-Young Yun, Minchan Jeong, Jun Hyun Nam, Jinwoo Shin, Richard Combes

TL;DR

Contextual linear bandits with noisy and missing features pose a mismatch between observed data and latent rewards that defeats standard realizability-based guarantees. The authors define a Bayesian oracle that selects actions based on the conditional distribution of the latent features given observed data, and show this oracle can differ substantially from the naive argmax of observed scores. To address this, they propose Contextual Linear Bandits on Bayesian Features (CLBBF), which estimates Bayesian features from data and uses OFUL-style optimism, achieving a regret of $R(T)=\tilde{O}\left(d\sqrt{T}+\frac{d^2}{p^{3/2}}\sqrt{\frac{T}{K}}+\frac{d}{p^4K}\right)$, reducing to $\tilde{O}(d\sqrt{T})$ when $K$ is large. Empirical results on synthetic and real-world data validate robustness to missing data and demonstrate practical relevance for privacy-preserving or noisy-feature settings.

Abstract

We study contextual linear bandit problems under feature uncertainty, where the features are noisy and have missing entries. To address the challenges posed by this noise, we analyze Bayesian oracles given the observed noisy features. Our Bayesian analysis reveals that the optimal hypothesis can significantly deviate from the underlying realizability function, depending on the noise characteristics. These deviations are highly non-intuitive and do not occur in classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. Therefore, we propose an algorithm that aims to approximate the Bayesian oracle based on the observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound when there is a large number of arms. We demonstrate the proposed algorithm using synthetic and real-world datasets.

Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles

TL;DR

Contextual linear bandits with noisy and missing features pose a mismatch between observed data and latent rewards that defeats standard realizability-based guarantees. The authors define a Bayesian oracle that selects actions based on the conditional distribution of the latent features given observed data, and show this oracle can differ substantially from the naive argmax of observed scores. To address this, they propose Contextual Linear Bandits on Bayesian Features (CLBBF), which estimates Bayesian features from data and uses OFUL-style optimism, achieving a regret of , reducing to when is large. Empirical results on synthetic and real-world data validate robustness to missing data and demonstrate practical relevance for privacy-preserving or noisy-feature settings.

Abstract

We study contextual linear bandit problems under feature uncertainty, where the features are noisy and have missing entries. To address the challenges posed by this noise, we analyze Bayesian oracles given the observed noisy features. Our Bayesian analysis reveals that the optimal hypothesis can significantly deviate from the underlying realizability function, depending on the noise characteristics. These deviations are highly non-intuitive and do not occur in classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. Therefore, we propose an algorithm that aims to approximate the Bayesian oracle based on the observed information under this model, achieving regret bound when there is a large number of arms. We demonstrate the proposed algorithm using synthetic and real-world datasets.

Paper Structure

This paper contains 15 sections, 14 theorems, 92 equations, 3 figures, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM STATEMENT
  3. WARM-UP: NOISY FEATURES WITHOUT MISSING ENTRIES
  4. NOISY FEATURES WITH MISSING ENTRIES
  5. NUMERICAL EXPERIMENTS
  6. DISCUSSION ON EXTENSIONS
  7. CONCLUSION
  8. APPENDIX
  9. Pseudocode of OFUL
  10. Proof of Proposition \ref{['cor:LinRel']}
  11. Proof of Theorem \ref{['thm:regret']}
  12. Tightness of the regret bound of Theorem \ref{['thm:regret']} with respect to $p$
  13. Algorithm \ref{['alg:alg2']} and its regret bound
  14. Regret bound of Algorithm \ref{['alg:alg2']}
  15. Real-world experiments

Key Result

Proposition 1

Define: Then the Bayesian oracle policy under the noisy features without missing entries selects arm $a^\star_t$ at time step $t\in[T]$ such that:

Figures (3)

  • Figure 1: A toy example
  • Figure 2: Performance of Algorithm \ref{['alg:alg1']} and OFUL: $R(T)$ versus missing probability $1-p$ with (a) $K=30$ and (b) $K=100$ in synthetic datasets; and $R(t)$ or CTR versus time step $t$ in (c) synthetic and (d) real-world datasets.
  • Figure 3: Click-through rate (CTR) for Algorithm \ref{['alg:alg1']}, OFUL, and random policy for real-world datasets.

Theorems & Definitions (26)

  • Proposition 1
  • proof
  • Proposition 2
  • Theorem 1
  • proof
  • Theorem 2
  • proof : Proof sketch
  • Lemma 1: Theorem 2 in abbasi2011improved
  • Lemma 2
  • proof
  • ...and 16 more