Interactive Machine Learning: From Theory to Scale

TL;DR

This work develops a cohesive theory of interactive machine learning across three axes: active learning with noisy data and rich models, scalable sequential decision making with large action spaces, and model selection under partial feedback. It introduces (i) abstention-enabled active learning that achieves polylogarithmic label complexity without low-noise assumptions and (ii) deep and nonparametric active learning guarantees that leverage neural-network expressiveness while maintaining efficiency. The results connect classical learning theory (Chow’s excess error, Massart/Tsybakov noise) with modern scalable algorithms, providing principled guidance for deploying interactive methods in large-scale settings. Collectively, the contributions advance both statistical optimality and computational practicality, illustrating how abstention and structured complexity measures (e.g., disagreement coefficients, eluder dimensions) yield near-optimal learning with far fewer labels in practice. These developments offer a solid theoretical foundation for deploying interactive learning in high-stakes and data-deliberate domains, including deep learning systems and large-context bandits.

Abstract

Machine learning has achieved remarkable success across a wide range of applications, yet many of its most effective methods rely on access to large amounts of labeled data or extensive online interaction. In practice, acquiring high-quality labels and making decisions through trial-and-error can be expensive, time-consuming, or risky, particularly in large-scale or high-stakes settings. This dissertation studies interactive machine learning, in which the learner actively influences how information is collected or which actions are taken, using past observations to guide future interactions. We develop new algorithmic principles and establish fundamental limits for interactive learning along three dimensions: active learning with noisy data and rich model classes, sequential decision making with large action spaces, and model selection under partial feedback. Our results include the first computationally efficient active learning algorithms achieving exponential label savings without low-noise assumptions; the first efficient, general-purpose contextual bandit algorithms whose guarantees are independent of the size of the action space; and the first tight characterizations of the fundamental cost of model selection in sequential decision making. Overall, this dissertation advances the theoretical foundations of interactive learning by developing algorithms that are statistically optimal and computationally efficient, while also providing principled guidance for deploying interactive learning methods in large-scale, real-world settings.

Figures (12)

• Figure 1: Illustration of decision regions under different error criteria. Top: standard excess error $\mathop{\mathrm{err}}\nolimits(\widehat{h}) - \mathop{\mathrm{err}}\nolimits(h^\star)$. Second: Chow's excess error $\mathop{\mathrm{err}}\nolimits_{\gamma}(\widehat{h}) - \mathop{\mathrm{err}}\nolimits(h^\star)$. Third: standard excess error $\mathop{\mathrm{err}}\nolimits(\widehat{h}) - \mathop{\mathrm{err}}\nolimits(h^\star)$ under Massart noise condition with parameter $\gamma$. Bottom: Chow's excess error relative to the optimal abstaining classifier, i.e., $\mathop{\mathrm{err}}\nolimits_{\gamma}(\widehat{h}) -\inf_{h: \mathcal{X} \rightarrow \{0, 1,\bot\}} \mathop{\mathrm{err}}\nolimits_{\gamma}(h)$. In this figure, positive corresponds to predicting label 1 and negative to predicting label 0.
• Figure 2: Performance of SpannerGreedy on amazon-3m.
• Figure 3: Comparison of regret on a bandit dataset with a discrete action space.
• Figure 4: Pareto optimal rates for bandit learning with multiple best arms.
• Figure 5: Experiments on synthetic dataset. (a) Comparison of regret with varying hardness level $\alpha$ (b) Comparison of progressive regret curve with $\alpha = 0.25$.

Theorems & Definitions (372)

• Theorem 2.2: Informal
• Theorem 2.3: Informal
• Proposition 2.4: Informal
• Theorem 2.5: Informal
• Definition 2.6: Value function disagreement coefficient
• Theorem 2.7
• Theorem 2.8
• Definition 2.9: Proper abstention
• Proposition 2.9
• Definition 2.10: Massart noise, massart2006risk