Partial Feedback Online Learning
Shihao Shao, Cong Fang, Zhouchen Lin, Dacheng Tao
TL;DR
The paper investigates partial-feedback online learning where each round supplies a set of valid labels but the learner observes only a single witness label. It introduces two key complexity measures—the Partial-Feedback Littlestone dimension ${\rm PFLdim}$ for deterministic learners and the Partial-Feedback Measure Shattering dimension ${\rm PMSdim}$ for randomized learners—to achieve near-complete minimax regret characterizations in the set-realizable regime. It proves tight upper and lower bounds: deterministic regret equals ${\rm PFL}_T(\tilde{\mathcal{H}})$ and randomized regret is governed by ${\rm PMS}_{(T,\gamma)}(\tilde{\mathcal{H}})$ with bounds that match up to constants, while identifying conditions (finite Helly number or nested-inclusion label structure) under which deterministic and randomized learnability separate. The work also extends to set-valued online learning, answering open questions about realizability and shedding light on the inherent hardness outside realizability, thereby motivating new noise-sensitive complexity notions for partial-information learning. Overall, it provides a principled, dimension-based theory for partial-feedback online learning with both deterministic and randomized predictors, highlighting fundamental limits and separations across feedback regimes.
Abstract
We study partial-feedback online learning, where each instance admits a set of correct labels, but the learner only observes one correct label per round; any prediction within the correct set is counted as correct. This model captures settings such as language generation, where multiple responses may be valid but data provide only a single reference. We give a near-complete characterization of minimax regret for both deterministic and randomized learners in the set-realizable regime, i.e., in the regime where sublinear regret is generally attainable. For deterministic learners, we introduce the Partial-Feedback Littlestone dimension (PFLdim) and show it precisely governs learnability and minimax regret; technically, PFLdim cannot be defined via the standard version space, requiring a new collection version space viewpoint and an auxiliary dimension used only in the proof. We further develop the Partial-Feedback Measure Shattering dimension (PMSdim) to obtain tight bounds for randomized learners. We identify broad conditions ensuring inseparability between deterministic and randomized learnability (e.g., finite Helly number or nested-inclusion label structure), and extend the argument to set-valued online learning, resolving an open question of Raman et al. [2024b]. Finally, we show a sharp separation from weaker realistic and agnostic variants: outside set realizability, the problem can become information-theoretically intractable, with linear regret possible even for $|H|=2$. This highlights the need for fundamentally new, noise-sensitive complexity measures to meaningfully characterize learnability beyond set realizability.