Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Double-Edged Sword of Behavioral Responses in Strategic Classification: Theory and User Studies

Raman Ebrahimi, Kristen Vaccaro, Parinaz Naghizadeh

TL;DR

A strategic classification model that considers behavioral biases in human responses to algorithms, and shows how misperceptions of a classifier can lead to different types of discrepancies between biased and rational agents’ responses, and identifies when behavioral agents over- or under-invest in different features.

Abstract

When humans are subject to an algorithmic decision system, they can strategically adjust their behavior accordingly (``game'' the system). While a growing line of literature on strategic classification has used game-theoretic modeling to understand and mitigate such gaming, these existing works consider standard models of fully rational agents. In this paper, we propose a strategic classification model that considers behavioral biases in human responses to algorithms. We show how misperceptions of a classifier (specifically, of its feature weights) can lead to different types of discrepancies between biased and rational agents' responses, and identify when behavioral agents over- or under-invest in different features. We also show that strategic agents with behavioral biases can benefit or (perhaps, unexpectedly) harm the firm compared to fully rational strategic agents. We complement our analytical results with user studies, which support our hypothesis of behavioral biases in human responses to the algorithm. Together, our findings highlight the need to account for human (cognitive) biases when designing AI systems, and providing explanations of them, to strategic human in the loop.

The Double-Edged Sword of Behavioral Responses in Strategic Classification: Theory and User Studies

TL;DR

A strategic classification model that considers behavioral biases in human responses to algorithms, and shows how misperceptions of a classifier can lead to different types of discrepancies between biased and rational agents’ responses, and identifies when behavioral agents over- or under-invest in different features.

Abstract

When humans are subject to an algorithmic decision system, they can strategically adjust their behavior accordingly (``game'' the system). While a growing line of literature on strategic classification has used game-theoretic modeling to understand and mitigate such gaming, these existing works consider standard models of fully rational agents. In this paper, we propose a strategic classification model that considers behavioral biases in human responses to algorithms. We show how misperceptions of a classifier (specifically, of its feature weights) can lead to different types of discrepancies between biased and rational agents' responses, and identify when behavioral agents over- or under-invest in different features. We also show that strategic agents with behavioral biases can benefit or (perhaps, unexpectedly) harm the firm compared to fully rational strategic agents. We complement our analytical results with user studies, which support our hypothesis of behavioral biases in human responses to the algorithm. Together, our findings highlight the need to account for human (cognitive) biases when designing AI systems, and providing explanations of them, to strategic human in the loop.

Paper Structure

This paper contains 22 sections, 6 theorems, 14 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Model and Preliminaries
  3. Agents' Strategic Responses
  4. Alternative Cost Functions
  5. Quadratic Cost Function.
  6. Weighted Manhattan Distance Cost Function.
  7. Cost Function in the User Studies.
  8. Firm's Response
  9. User Study
  10. Study Design, Participants, and Setup
  11. Results and Discussion
  12. Conclusion
  13. Additional related work
  14. Proofs
  15. Proof of Lemma \ref{['lemma:band-optimization']}, Lemma \ref{['lemma:quad-cost-band']}, and Lemma \ref{['lemma:manhattan-cost-band']}
  16. ...and 7 more sections

Key Result

Lemma 1

Let $d({\bm{x}}_0, {\bm{\theta}}, \theta_0)=\frac{\theta_0-{\bm{\theta}}^T{\bm{x}}_0}{\norm{{\bm{\theta}}}_2}$ denote ${\bm{x}}_0$'s distance to the hyperplane ${\bm{\theta}}^T{\bm{x}}=\theta_0$. Then, for an agent with starting feature vector ${\bm{x}}_0$, if $0 < d({\bm{x}}_0, {\bm{\theta}}, \thet Otherwise, ${\bm{x}}_\text{NB} = {\bm{x}}_0$. For behaviorally biased agents, ${\bm{x}}_{B}$ is obt

Figures (8)

  • Figure 1: (a) Fully rational and (b) Biased responses, and (c) Classes of differing actions under quadratic costs.
  • Figure 2: Strategic responses under quadratic costs.
  • Figure 3: Strategic responses under Manhattan costs.
  • Figure 4: Strategic responses under a quadratic cost (green) vs. a piece-wise linear cost function (blue).
  • Figure 5: An oblivious firm may have lower (top) or higher (middle) utility when agents are biased (vs. rational). A non-oblivious firm may still have a lower utility when agents are biased (bottom).
  • ...and 3 more figures

Theorems & Definitions (8)

  • Lemma 1
  • Proposition 1
  • Lemma 2
  • Lemma 3
  • Example 1
  • Proposition 2
  • Lemma 4
  • proof