Adversarial Resilience in Sequential Prediction via Abstention
Surbhi Goel, Steve Hanneke, Shay Moran, Abhishek Shetty
TL;DR
The paper introduces a beyond-worst-case framework for sequential prediction with abstention, where an adversary can inject clean-label examples but the learner may abstain without cost on those rounds. It shows that, when the marginal distribution of non-adversarial data is available, misclassification error scales with the VC dimension $d$ (not the Littlestone dimension) and provides concrete bounds such as $O(d^2\log T)$ for general VC classes, with stronger results for VC=1 and axis-aligned rectangles. It also develops structure-based and higher-order disagreement strategies, including a novel uncertainty measure based on shattering probabilities $\rho_k$, to achieve finite-sample guarantees in both known and unknown distribution settings. The framework connects to testable learning and uncertainty quantification, offering practical pathways for robust sequential prediction in high-stakes settings where abstention can prevent costly misclassifications.
Abstract
We study the problem of sequential prediction in the stochastic setting with an adversary that is allowed to inject clean-label adversarial (or out-of-distribution) examples. Algorithms designed to handle purely stochastic data tend to fail in the presence of such adversarial examples, often leading to erroneous predictions. This is undesirable in many high-stakes applications such as medical recommendations, where abstaining from predictions on adversarial examples is preferable to misclassification. On the other hand, assuming fully adversarial data leads to very pessimistic bounds that are often vacuous in practice. To capture this motivation, we propose a new model of sequential prediction that sits between the purely stochastic and fully adversarial settings by allowing the learner to abstain from making a prediction at no cost on adversarial examples. Assuming access to the marginal distribution on the non-adversarial examples, we design a learner whose error scales with the VC dimension (mirroring the stochastic setting) of the hypothesis class, as opposed to the Littlestone dimension which characterizes the fully adversarial setting. Furthermore, we design a learner for VC dimension~1 classes, which works even in the absence of access to the marginal distribution. Our key technical contribution is a novel measure for quantifying uncertainty for learning VC classes, which may be of independent interest.