Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Identifying and Improving Disability Bias in GPT-Based Resume Screening

Kate Glazko, Yusuf Mohammed, Ben Kosa, Venkatesh Potluri, Jennifer Mankoff

TL;DR

This study quantifies disability bias in GPT-based resume screening by comparing a control CV with disability-enhanced CVs across seven disability types using GPT-4 and a Disability-Aware GPT (DA-GPT). It combines a resume audit with both quantitative rankings and qualitative analyses to reveal direct and indirect ableist reasoning in GPT-4’s justifications. The results show measurable bias against disability-enhanced resumes, but demonstrate that training a DA-GPT with disability justice principles significantly reduces bias, especially for Deaf and general disability cases. The work highlights the need for stress-testing and ethical deployment of AI in hiring, and provides concrete directions for improving fairness in GAI-based recruiting tools while acknowledging remaining gaps and limitations.

Abstract

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.

Identifying and Improving Disability Bias in GPT-Based Resume Screening

TL;DR

This study quantifies disability bias in GPT-based resume screening by comparing a control CV with disability-enhanced CVs across seven disability types using GPT-4 and a Disability-Aware GPT (DA-GPT). It combines a resume audit with both quantitative rankings and qualitative analyses to reveal direct and indirect ableist reasoning in GPT-4’s justifications. The results show measurable bias against disability-enhanced resumes, but demonstrate that training a DA-GPT with disability justice principles significantly reduces bias, especially for Deaf and general disability cases. The work highlights the need for stress-testing and ethical deployment of AI in hiring, and provides concrete directions for improving fairness in GAI-based recruiting tools while acknowledging remaining gaps and limitations.

Abstract

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.
Paper Structure (34 sections, 3 figures, 5 tables)

This paper contains 34 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI/ML Hiring Tools and Bias
  4. Resume Audit Studies for Uncovering Bias
  5. Summary and Open Questions
  6. Methods
  7. CV and ECV Creation
  8. GAI Selection and Preparation
  9. Creating a Disability-Aware GPT
  10. Prompt Engineering
  11. Data Collection Method
  12. Data Analysis
  13. Quantitative Analysis
  14. Qualitative Analysis
  15. Limitations
  16. ...and 19 more sections

Figures (3)

  • Figure 1: (Left) Comparison of the number of times the disability-mentioning CV was top choice with DA-GPT trials (forward, polka-dot bar) and GPT-4 trials (rear, solid bar) in each condition. (Right) Word count of frequent words in GPT-4 trials with ECV (forward, polka-dot bar) and CV (rear, solid bar).*Denotes statistically significance difference p<0.05, ** at p<0.01, *** at p<0.001
  • Figure 2: Control (CV) Resume Representation
  • Figure 3: Enhanced (ECV) Resume with Disability Representation