Oracle-Efficient Hybrid Online Learning with Unknown Distribution
Changlong Wu, Jin Sima, Wojciech Szpankowski
TL;DR
The paper tackles oracle-efficient online learning when features come from an unknown i.i.d. distribution and labels are chosen adversarially. It introduces an epoch-based relaxation framework that uses an ERM oracle and hallucinated samples drawn from an estimated distribution to achieve sublinear regret without access to the feature sampler, and proves bounds that depend on the hypothesis class complexity via Rademacher complexity and fat-shattering dimensions. The main results show $ ilde{O}(T^{3/4})$ regret for finite VC classes and $ ilde{O}(T^{(p+1)/(p+2)})$ for $ ext{fat}_ ext{shattering}(\alpha) hicksim inom{ ext{alpha}^{-p}}$, along with extensions to shifting distributions $ ilde{O}(T^{4/5}K^{1/5})$ and contextual K-armed bandits $ ilde{O}((K^{2/3}(\log|mathcal{H}|)^{1/3} + K oot 2 ext{ log }K) imes T^{4/5})$. The approach avoids requiring a sampling oracle for the feature process by introducing approx-admissibility and a careful symmetrization argument, yielding first known oracle-efficient sublinear regrets in this hybrid setting and validating a conjecture by Lazaric and Munos. The results extend to special classes, non-stationary settings, and contextual bandits, highlighting the broad applicability of the relaxation-based, oracle-efficient paradigm in uncertain data-generating environments.
Abstract
We study the problem of oracle-efficient hybrid online learning when the features are generated by an unknown i.i.d. process and the labels are generated adversarially. Assuming access to an (offline) ERM oracle, we show that there exists a computationally efficient online predictor that achieves a regret upper bounded by $\tilde{O}(T^{\frac{3}{4}})$ for a finite-VC class, and upper bounded by $\tilde{O}(T^{\frac{p+1}{p+2}})$ for a class with $α$ fat-shattering dimension $α^{-p}$. This provides the first known oracle-efficient sublinear regret bounds for hybrid online learning with an unknown feature generation process. In particular, it confirms a conjecture of Lazaric and Munos (JCSS 2012). We then extend our result to the scenario of shifting distributions with $K$ changes, yielding a regret of order $\tilde{O}(T^{\frac{4}{5}}K^{\frac{1}{5}})$. Finally, we establish a regret of $\tilde{O}((K^{\frac{2}{3}}(\log|\mathcal{H}|)^{\frac{1}{3}}+K)\cdot T^{\frac{4}{5}})$ for the contextual $K$-armed bandits with a finite policy set $\mathcal{H}$, i.i.d. generated contexts from an unknown distribution, and adversarially generated costs.