Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Decision Subjects' Engagement with and Perceived Fairness of AI Models When Opportunities of Qualification Improvement Exist

Meric Altug Gemalmaz, Ming Yin

TL;DR

The paper empirically examines how AI decision fairness influences decision subjects' engagement and fairness perceptions under repeated, strategic interactions, including the possibility to improve one's qualification. Through three MTurk experiments centered on simulated loan decisions, subjects could continue applying and invest in credit-score improvements across fair and biased AI treatments. Across studies, engagement metrics (improvement and retention) were largely insensitive to fairness properties, while perceived fairness varied with group bias, particularly under harder-to-improve scenarios and gender-based biases. The findings imply that apparent fairness in engagement should not be used as a proxy for AI fairness, and highlight the potential of providing recourse or qualification-improvement guidance as a temporary measure to maintain retention while addressing fundamental fairness issues.

Abstract

We explore how an AI model's decision fairness affects people's engagement with and perceived fairness of the model if they are subject to its decisions, but could repeatedly and strategically respond to these decisions. Two types of strategic responses are considered -- people could determine whether to continue interacting with the model, and whether to invest in themselves to improve their chance of future favorable decisions from the model. Via three human-subject experiments, we found that in decision subjects' strategic, repeated interactions with an AI model, the model's decision fairness does not change their willingness to interact with the model or to improve themselves, even when the model exhibits unfairness on salient protected attributes. However, decision subjects still perceive the AI model to be less fair when it systematically biases against their group, especially if the difficulty of improving one's qualification for the favorable decision is larger for the lowly-qualified people.

Understanding Decision Subjects' Engagement with and Perceived Fairness of AI Models When Opportunities of Qualification Improvement Exist

TL;DR

The paper empirically examines how AI decision fairness influences decision subjects' engagement and fairness perceptions under repeated, strategic interactions, including the possibility to improve one's qualification. Through three MTurk experiments centered on simulated loan decisions, subjects could continue applying and invest in credit-score improvements across fair and biased AI treatments. Across studies, engagement metrics (improvement and retention) were largely insensitive to fairness properties, while perceived fairness varied with group bias, particularly under harder-to-improve scenarios and gender-based biases. The findings imply that apparent fairness in engagement should not be used as a proxy for AI fairness, and highlight the potential of providing recourse or qualification-improvement guidance as a temporary measure to maintain retention while addressing fundamental fairness issues.

Abstract

We explore how an AI model's decision fairness affects people's engagement with and perceived fairness of the model if they are subject to its decisions, but could repeatedly and strategically respond to these decisions. Two types of strategic responses are considered -- people could determine whether to continue interacting with the model, and whether to invest in themselves to improve their chance of future favorable decisions from the model. Via three human-subject experiments, we found that in decision subjects' strategic, repeated interactions with an AI model, the model's decision fairness does not change their willingness to interact with the model or to improve themselves, even when the model exhibits unfairness on salient protected attributes. However, decision subjects still perceive the AI model to be less fair when it systematically biases against their group, especially if the difficulty of improving one's qualification for the favorable decision is larger for the lowly-qualified people.
Paper Structure (32 sections, 10 figures, 9 tables)

This paper contains 32 sections, 10 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Study 1
  4. Experimental Design
  5. Tasks.
  6. Treatments.
  7. Procedure.
  8. Analysis Methods
  9. Experimental Results
  10. RQ1: Impacts of AI's decision fairness on engagement.
  11. RQ2: Impacts of AI's decision fairness on fairness perceptions.
  12. Study 2
  13. Experimental Design
  14. Experimental Results
  15. Re-examining RQ1: Impacts of AI's decision fairness on engagement.
  16. ...and 17 more sections

Figures (10)

  • Figure 1: An illustration of the process of the loan application task. Here, $c$ is the subject's current qualification level, while $c'$ denotes the qualification level after an improvement attempt, which can either remain the same or advance to the next level.
  • Figure A1: An example of the flowchart that subjects in the unfair AI treatment saw in the experiment, which summarizes the AI model's decisions on different groups of applicants in the past round. Subjects could see the frequency of the AI model approving/denying loans both for applicants with/without "high" credit scores (i.e., a score of at least 650), and for applicants with similar credit scores as themselves (i.e., applicants with the same credit scores as themselves or one level above/below themselves).
  • Figure A2: Distributions of (a) the number of improvement attempts that subjects made, and (b) the number of rounds that subjects interacted with the AI model, for subjects who were assigned to the fair AI model treatment, the red (and advantaged) group of the unfair AI model treatment, and the blue (and disadvantaged) group of the unfair AI model treatment, in Study 1. Curves represent the probability density functions obtained through kernel density estimation.
  • Figure A3: Distributions of (a) the number of improvement attempts that subjects made, and (b) the number of rounds that subjects interacted with the AI model, for subjects who were assigned to the fair AI model treatment, the red (and advantaged) group of the unfair AI model treatment, and the blue (and disadvantaged) group of the unfair AI model treatment, in Study 2 sub-experiment "Easy to hard". Curves represent the probability density functions obtained through kernel density estimation.
  • Figure A4: Distributions of (a) the number of improvement attempts that subjects made, and (b) the number of rounds that subjects interacted with the AI model, for subjects who were assigned to the fair AI model treatment, the red (and advantaged) group of the unfair AI model treatment, and the blue (and disadvantaged) group of the unfair AI model treatment, in Study 2 sub-experiment "Hard to easy". Curves represent the probability density functions obtained through kernel density estimation.
  • ...and 5 more figures