Classification Under Strategic Self-Selection

TL;DR

This work studies the effects of self-selection on learning, the implications of learning on the composition of the self-selected population, and proposes a differentiable framework for learning under self-selective behavior, which can be optimized effectively.

Abstract

When users stand to gain from certain predictions, they are prone to act strategically to obtain favorable predictive outcomes. Whereas most works on strategic classification consider user actions that manifest as feature modifications, we study a novel setting in which users decide -- in response to the learned classifier -- whether to at all participate (or not). For learning approaches of increasing strategic awareness, we study the effects of self-selection on learning, and the implications of learning on the composition of the self-selected population. We then propose a differentiable framework for learning under self-selective behavior, which can be optimized effectively. We conclude with experiments on real data and simulated behavior that both complement our analysis and demonstrate the utility of our approach.

Figures (10)

• Figure 1: The application process. Candidates who apply must first pass a screening classifier; if successful, they advance to take a costly qualifying test. Candidates are strategic, and apply only if it is cost-effective. Since their likelihood of passing screening depends on the classifier (through its conditional precision on past data), learning has the power to shape the composition of the applicant population.
• Figure 2: ${\mathtt{prc}}_z$ under optimal $f^*$ for high and low base rates.
• Figure 3: Self-selective distributions. Absent strategic behavior, all groups in the population are assumed to participate (left). But when applications depend on the learned classifier (here, $f_1$ vs. $f_2$), self-selection shapes the target distribution that the classifier will face (center vs. right).
• Figure 4: Precision and application. For a fixed score function ${\phi}$ and varying threshold ${\tau}$, different groups ($K=10$, colored lines) exhibit different precision curves. Though ${\phi}$ is not mutually calibrated, most curves are roughly monotone and cross the cost ${c}$ (dashed line) at most once. This induces an ordering $\preceq_{\phi}$ over applications $a^*$ (lower plot). A semi-strategic learner can affect $a^*$ only by thresholding on $\preceq_{\phi}$.
• Figure 5: Social cost per cost $c$ for different methods.

Theorems & Definitions (29)

• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Corollary 1
• Definition 1: Calibrated score function
• Lemma 1
• Corollary 2
• Corollary 3
• Corollary 4