The Active and Noise-Tolerant Strategic Perceptron

TL;DR

This work develops an active learning algorithm for learning linear separators in the presence of strategically manipulating agents, achieving substantial label-efficiency gains even under nonrealizable data. By adapting the Active Perceptron with a cost-aware prediction threshold ($1/c$), restricting label queries to unmanipulated negatives within a carefully defined region, and normalizing updates, the authors reduce strategic learning to a nonstrategic framework and obtain tilde $O(d \log(1/\varepsilon))$ label queries and comparable mistake bounds in the realizable case. In the noisy (nonrealizable) setting, they show an excess error of $\Theta(\varepsilon)$ with similar label complexity, provided a fraction $\tilde{\Omega}(\varepsilon)$ of inputs are inverted, addressing an open question in strategic classification. The approach yields a computationally efficient algorithm with strong guarantees, enabling robust, label-efficient learning in domains where agents can manipulate observed features.

Abstract

We initiate the study of active learning algorithms for classifying strategic agents. Active learning is a well-established framework in machine learning in which the learner selectively queries labels, often achieving substantially higher accuracy and efficiency than classical supervised methods-especially in settings where labeling is costly or time-consuming, such as hiring, admissions, and loan decisions. Strategic classification, however, addresses scenarios where agents modify their features to obtain more favorable outcomes, resulting in observed data that is not truthful. Such manipulation introduces challenges beyond those in learning from clean data. Our goal is to design active and noise-tolerant algorithms that remain effective in strategic environments-algorithms that classify strategic agents accurately while issuing as few label requests as possible. The central difficulty is to simultaneously account for strategic manipulation and preserve the efficiency gains of active learning. Our main result is an algorithm for actively learning linear separators in the strategic setting that preserves the exponential improvement in label complexity over passive learning previously obtained only in the non-strategic case. Specifically, for data drawn uniformly from the unit sphere, we show that a modified version of the Active Perceptron algorithm [DKM05,YZ17] achieves excess error $ε$ using only $\tilde{O}(d \ln \frac{1}ε)$ label queries and incurs at most $\tilde{O}(d \ln \frac{1}ε)$ additional mistakes relative to the optimal classifier, even in the nonrealizable case, when a $\tildeΩ(ε)$ fraction of inputs have inconsistent labels with the optimal classifier. The algorithm is computationally efficient and, under these distributional assumptions, requires substantially fewer label queries than prior work on strategic Perceptron [ABBN21].

Figures (1)

• Figure 1: Illustration of observed regions of examples, where the original examples are on the surface of a unit ball, as assumed in prior literature on active learning Dasgupta2005AnalysisDBLP:conf/nips/YanZ17. The colored area shows the potential positions where the examples can be observed, following \ref{['lm:strategic_action']}. Under this assumption, there is a one-to-one mapping between the original examples and observed ones, and observing an example $\bm{x}_i$ exactly identifies the original position $\bm{z}_i$. For instance, when observing $\bm{x}_1$ to recover the original point $\bm{z}_1$, one needs to consider the projection of $\bm{x}_1$ on the surface of the pall, in the opposite direction of $\bm{v}$. This one-to-one mapping no longer exists under the relaxed assumptions that original examples can be inside the ball and not just the surface.

Theorems & Definitions (18)

• Definition 1: $\hat{\bm{x}}$
• Theorem 2
• Lemma 2: Strategic Action
• Corollary 3
• Lemma 4: DBLP:conf/nips/YanZ17 Lemma 1
• Lemma 5
• proof
• Corollary 6
• Lemma 7
• proof