Optimal Mistake Bounds for Transductive Online Learning

Zachary Chase; Steve Hanneke; Shay Moran; Jonathan Shafer

Paper

Optimal Mistake Bounds for Transductive Online Learning

Abstract

We resolve a 30-year-old open problem concerning the power of unlabeled data in online learning by tightly quantifying the gap between transductive and standard online learning. In the standard setting, the optimal mistake bound is characterized by the Littlestone dimension

of the concept class

(Littlestone 1987). We prove that in the transductive setting, the mistake bound is at least

. This constitutes an exponential improvement over previous lower bounds of

, and

, due respectively to Ben-David, Kushilevitz, and Mansour (1995, 1997) and Hanneke, Moran, and Shafer (2023). We also show that this lower bound is tight: for every

, there exists a class of Littlestone dimension

with transductive mistake bound

. Our upper bound also improves upon the best known upper bound of

from Ben-David, Kushilevitz, and Mansour (1997). These results establish a quadratic gap between transductive and standard online learning, thereby highlighting the benefit of advance access to the unlabeled instance sequence. This contrasts with the PAC setting, where transductive and standard learning exhibit similar sample complexities.