Linear Bandits on Ellipsoids: Minimax Optimal Algorithms

TL;DR

This work tackles stochastic linear bandits with ellipsoidal action sets and establishes minimax optimality via a new information-theoretic lower bound and a novel non-optimistic algorithm, E2TC, that is computationally efficient. E2TC combines a norm-estimation warmup, an axis-focused exploration phase, and a commit phase that greedily exploits the learned parameter, achieving a regret of $R_T(\theta)=O(\min(\|\theta\|_{A}T, d\sigma\sqrt{T}+d\|\theta\|_{A}))$ and running in $O(dT+d^2\log(T/d)+d^3)$ time with $O(d^2)$ memory. The paper also proves a locally asymptotically minimax optimal guarantee, extends to non-centered ellipsoids via a reduction, and provides numerical experiments showing superior computational efficiency and competitive regret against optimistic baselines. These results imply that minimax-optimal performance need not come at the cost of tractable computation in structured bandit settings. The findings advance understanding of structure-aware bandit algorithms and offer a practical approach for high-dimensional, ellipsoid-constrained decision problems.

Abstract

We consider linear stochastic bandits where the set of actions is an ellipsoid. We provide the first known minimax optimal algorithm for this problem. We first derive a novel information-theoretic lower bound on the regret of any algorithm, which must be at least $Ω(\min(d σ\sqrt{T} + d \|θ\|_{A}, \|θ\|_{A} T))$ where $d$ is the dimension, $T$ the time horizon, $σ^2$ the noise variance, $A$ a matrix defining the set of actions and $θ$ the vector of unknown parameters. We then provide an algorithm whose regret matches this bound to a multiplicative universal constant. The algorithm is non-classical in the sense that it is not optimistic, and it is not a sampling algorithm. The main idea is to combine a novel sequential procedure to estimate $\|θ\|$, followed by an explore-and-commit strategy informed by this estimate. The algorithm is highly computationally efficient, and a run requires only time $O(dT + d^2 \log(T/d) + d^3)$ and memory $O(d^2)$, in contrast with known optimistic algorithms, which are not implementable in polynomial time. We go beyond minimax optimality and show that our algorithm is locally asymptotically minimax optimal, a much stronger notion of optimality. We further provide numerical experiments to illustrate our theoretical findings.

Figures (4)

• Figure 1: Schematic illustration of the differences between minimax and locally asymptotically minimax lower bounds. Minimax lower bounds require that the regret curve of any algorithm goes above the bound at least once, whereas locally minimax lower bounds require that the curve of any algorithm goes above the bound in any region of size $\delta_T = o(1)$. In both figures, Algorithms 1 and 2 match the lower bound, and Algorithm 3 does not.
• Figure 2: Comparison of the final regret of E2TC, OFUL and OLSOFUL for different norms.
• Figure 3: Final regret of E2TC, OFUL and OLSOFUL as a function of the dimension $d$.
• Figure 4: Runtime comparison of E2TC and OFUL for different norms and dimensions.

Theorems & Definitions (23)

• Theorem 2.1
• Proposition 2.1
• Remark 3.1: Computational complexity
• Lemma 3.1
• Lemma 3.2: Regret of Phase 1
• Lemma 3.3: Concentration of error in linear regression
• Theorem 3.1
• Corollary 3.1
• proof
• Theorem 3.2