A Combinatorial Characterization of Supervised Online Learnability
Vinod Raman, Unique Subedi, Ambuj Tewari
TL;DR
This work introduces the sequential minimax dimension (SMdim), a scale-sensitive combinatorial measure that characterizes online learnability for arbitrary bounded losses. It provides a generic online learner whose regret is governed by SMdim and proves a tight, order-optimal minimax bound tying regret to SMdim across scales, while unifying existing dimensions such as Littlestone’s $L$-dimension and sequential fat-shattering via Helly-type geometry. The authors also relate SMdim to finite-width tree dimensions through the Helly-number framework, deriving reductions to known dimensions in standard settings and establishing tight upper and lower bounds (up to logarithmic factors) for minimax regret. These results yield a comprehensive, general theory for online supervised learning, applicable across classification, regression, and structured-label problems, and highlight the role of geometry (Helly spaces) in translating infinite-width definitions into finite, computable characterizations.
Abstract
We study the online learnability of hypothesis classes with respect to arbitrary, but bounded loss functions. No characterization of online learnability is known at this level of generality. We give a new scale-sensitive combinatorial dimension, named the sequential minimax dimension, and show that it gives a tight quantitative characterization of online learnability. In addition, we show that the sequential minimax dimension subsumes most existing combinatorial dimensions in online learning theory.