Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Navigating Sparsities in High-Dimensional Linear Contextual Bandits

Rui Zhao, Zihan Chen, Zemin Zheng

TL;DR

This work tackles high-dimensional linear contextual bandits by identifying two sparsity structures—sparse model parameters and sparse eigenvalues of context covariances—and introducing a flexible PointWise Estimator (PWE) to adaptively handle both. Building on PWE, the HOPE algorithm employs an Explore-Then-Commit scheme to achieve sublinear regret across four heterogeneous scenarios, integrating Lasso and RDL as initial estimators where appropriate. Theoretical analyses provide a universal regret bound and detailed rates for each scenario, demonstrating HOPE’s ability to match or surpass prior methods in homogeneous settings and to excel in mixed sparsity contexts where existing approaches fail. Empirical results corroborate these findings, showing HOPE’s robust performance across diverse covariance and sparsity regimes and confirming its practical significance for high-dimensional contextual decision-making.

Abstract

High-dimensional linear contextual bandit problems remain a significant challenge due to the curse of dimensionality. Existing methods typically consider either the model parameters to be sparse or the eigenvalues of context covariance matrices to be (approximately) sparse, lacking general applicability due to the rigidity of conventional reward estimators. To overcome this limitation, a powerful pointwise estimator is introduced in this work that adaptively navigates both kinds of sparsity. Based on this pointwise estimator, a novel algorithm, termed HOPE, is proposed. Theoretical analyses demonstrate that HOPE not only achieves improved regret bounds in previously discussed homogeneous settings (i.e., considering only one type of sparsity) but also, for the first time, efficiently handles two new challenging heterogeneous settings (i.e., considering a mixture of two types of sparsity), highlighting its flexibility and generality. Experiments corroborate the superiority of HOPE over existing methods across various scenarios.

Navigating Sparsities in High-Dimensional Linear Contextual Bandits

TL;DR

This work tackles high-dimensional linear contextual bandits by identifying two sparsity structures—sparse model parameters and sparse eigenvalues of context covariances—and introducing a flexible PointWise Estimator (PWE) to adaptively handle both. Building on PWE, the HOPE algorithm employs an Explore-Then-Commit scheme to achieve sublinear regret across four heterogeneous scenarios, integrating Lasso and RDL as initial estimators where appropriate. Theoretical analyses provide a universal regret bound and detailed rates for each scenario, demonstrating HOPE’s ability to match or surpass prior methods in homogeneous settings and to excel in mixed sparsity contexts where existing approaches fail. Empirical results corroborate these findings, showing HOPE’s robust performance across diverse covariance and sparsity regimes and confirming its practical significance for high-dimensional contextual decision-making.

Abstract

High-dimensional linear contextual bandit problems remain a significant challenge due to the curse of dimensionality. Existing methods typically consider either the model parameters to be sparse or the eigenvalues of context covariance matrices to be (approximately) sparse, lacking general applicability due to the rigidity of conventional reward estimators. To overcome this limitation, a powerful pointwise estimator is introduced in this work that adaptively navigates both kinds of sparsity. Based on this pointwise estimator, a novel algorithm, termed HOPE, is proposed. Theoretical analyses demonstrate that HOPE not only achieves improved regret bounds in previously discussed homogeneous settings (i.e., considering only one type of sparsity) but also, for the first time, efficiently handles two new challenging heterogeneous settings (i.e., considering a mixture of two types of sparsity), highlighting its flexibility and generality. Experiments corroborate the superiority of HOPE over existing methods across various scenarios.

Paper Structure

This paper contains 47 sections, 20 theorems, 106 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. The Previously Adopted Estimators
  4. A Powerful Pointwise Estimator
  5. Estimating the Support Set and the Initial Estimator
  6. Transforming the Linear Model
  7. Sparsifying the Nuisance Vector
  8. The Overall PWE Procedure
  9. The HOPE Algorithm
  10. Theoretical Analysis
  11. Scenario 1: Sparse Model Parameters
  12. Scenario 2: (Approximately) Sparse Eigenvalues of Context Covariance Matrices
  13. Scenario 3: Both Sparsities
  14. Scenario 4: Mixed Sparsities
  15. Experiments
  16. ...and 32 more sections

Key Result

Theorem 1

Under Assumps. aspt:problem_parameters, aspt:prediction_error and aspt:support, with an exploration phase lasting $T_0 = 2NK \leq T$ steps and $\lambda_N^{(i)} \asymp \sigma \sqrt{\log(N)/N}$ in Eqn. (eqn:PWE) for all arms $i\in [K]$, the regret of the HOPE algorithm is bounded as where $\mathrm{polylog}$ denotes a polynomial term in the logarithm of the input, and $G_{\mathcal{S}_1, \hat{\boldsy

Figures (3)

  • Figure 1: Applicability of previous works and HOPE, where the third regent, marked gray, is in general non-solvable.
  • Figure 2: Comparison of methods on four different cases. Smaller regret indicates better performance. The solid lines are the mean of 10 repetitions, and the bands represent the standard deviation.
  • Figure : PWE for arm $i$

Theorems & Definitions (53)

  • Example 1
  • Remark 1
  • Remark 2
  • Theorem 1
  • Proposition 1: Sparse Model Parameters
  • Remark 3
  • Proposition 2: Sparse Eigenvalues of $\boldsymbol{\Sigma}$: Example \ref{['example']}(A)
  • Proposition 3: Both Sparsities
  • Remark 4: Comparison within Scenario 3
  • Proposition 4: Mixed sparsity
  • ...and 43 more