Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Meta Learning in Bandits within Shared Affine Subspaces

Steven Bilaj, Sofien Dhouib, Setareh Maghsudi

TL;DR

The paper addresses meta-learning across multiple contextual linear bandit tasks by assuming task parameters concentrate around a low-dimensional affine subspace of dimension $p$, learned online via CCIPCA. It introduces two subspace-aware policies—LinUCB with projection bias and projection-based Linear Thompson Sampling—and provides regret bounds that quantify improvements when the subspace is correctly identified. Theoretical results show that the effective dimension reduces to $p$ and the regret scales with terms involving $p$, the residual dimension $q$, and the eigengap, while empirical experiments on synthetic and MovieLens data validate substantial performance gains over standard baselines. This approach offers a principled, interpretable way to accelerate learning across related bandit tasks by exploiting shared subspace structure with online subspace learning.

Abstract

We study the problem of meta-learning several contextual stochastic bandits tasks by leveraging their concentration around a low-dimensional affine subspace, which we learn via online principal component analysis to reduce the expected regret over the encountered bandits. We propose and theoretically analyze two strategies that solve the problem: One based on the principle of optimism in the face of uncertainty and the other via Thompson sampling. Our framework is generic and includes previously proposed approaches as special cases. Besides, the empirical results show that our methods significantly reduce the regret on several bandit tasks.

Meta Learning in Bandits within Shared Affine Subspaces

TL;DR

The paper addresses meta-learning across multiple contextual linear bandit tasks by assuming task parameters concentrate around a low-dimensional affine subspace of dimension , learned online via CCIPCA. It introduces two subspace-aware policies—LinUCB with projection bias and projection-based Linear Thompson Sampling—and provides regret bounds that quantify improvements when the subspace is correctly identified. Theoretical results show that the effective dimension reduces to and the regret scales with terms involving , the residual dimension , and the eigengap, while empirical experiments on synthetic and MovieLens data validate substantial performance gains over standard baselines. This approach offers a principled, interpretable way to accelerate learning across related bandit tasks by exploiting shared subspace structure with online subspace learning.

Abstract

We study the problem of meta-learning several contextual stochastic bandits tasks by leveraging their concentration around a low-dimensional affine subspace, which we learn via online principal component analysis to reduce the expected regret over the encountered bandits. We propose and theoretically analyze two strategies that solve the problem: One based on the principle of optimism in the face of uncertainty and the other via Thompson sampling. Our framework is generic and includes previously proposed approaches as special cases. Besides, the empirical results show that our methods significantly reduce the regret on several bandit tasks.
Paper Structure (20 sections, 14 theorems, 73 equations, 4 figures, 1 table, 3 algorithms)

This paper contains 20 sections, 14 theorems, 73 equations, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Subspace Learning
  5. Projection Meta learning with LinUCB
  6. Basics of LinUCB
  7. LinUCB with Projection Bias
  8. Analysis
  9. Projection Meta Learning with Linear Thompson Sampling
  10. Basics of Linear Thompson Sampling
  11. Thompson Sampling with Linear Payoffs within an Affine Subspace
  12. Analysis
  13. Algorithms
  14. Numerical Experiments
  15. Synthetic Data Experiments
  16. ...and 5 more sections

Key Result

Lemma 1

Let $\tau$ be a stopping time with respect to a filtration $\{\mathcal{F}_k\}_{k=1}^{\infty}$ and define $\boldsymbol{\eta}_k={\bf D}^\top_k\boldsymbol{\epsilon}$, with $\boldsymbol{\epsilon} \in {\mathbb R}^k$ as subgaussian noise vector. Then, for every $\delta\in(0,1)$, with probability at least

Figures (4)

  • Figure 1: Sample of task parameters (blue points) from a distribution with low variance along one dimension.
  • Figure 2: Expected transfer and total cumulative regret plots of the LinUCB and Thompson sampling methods compared to their projection counterparts and additional baselines.
  • Figure 3:
  • Figure 4: Expected transfer regret and total regret plots of our algorithms and baselines applied to the MovieLens data set. We have included 400 users in the simulations.

Theorems & Definitions (29)

  • Remark 1
  • Lemma 1: Self-normalized bound for vector-valued martingales
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • Remark 2
  • Definition 1
  • Definition 2
  • Lemma 5
  • ...and 19 more