Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users

Elinor Poole-Dayan, Deb Roy, Jad Kabbara

TL;DR

The paper investigates whether state-of-the-art LLMs exhibit targeted underperformance based on user demographics, focusing on English proficiency, education level, and country of origin. Using bios paired with prompts, three LLMs (GPT-4, Claude 3 Opus, Llama 3) are evaluated on TruthfulQA and SciQ to measure accuracy, truthfulness, and refusals, revealing disproportionate deficits for non-native, less educated, and non-US users, with strongest effects at trait intersections. The findings highlight substantial risks of biased information access and condescending behavior in personalized AI and emphasize the need for alignment and fairness strategies when deploying LLMs at scale. The work underscores practical implications for equitable information provision and motivates further research into mitigating demographic biases in LLM-powered systems.

Abstract

While state-of-the-art large language models (LLMs) have shown impressive performance on many tasks, there has been extensive research on undesirable model behavior such as hallucinations and bias. In this work, we investigate how the quality of LLM responses changes in terms of information accuracy, truthfulness, and refusals depending on three user traits: English proficiency, education level, and country of origin. We present extensive experimentation on three state-of-the-art LLMs and two different datasets targeting truthfulness and factuality. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US, rendering these models unreliable sources of information towards their most vulnerable users.

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users

TL;DR

The paper investigates whether state-of-the-art LLMs exhibit targeted underperformance based on user demographics, focusing on English proficiency, education level, and country of origin. Using bios paired with prompts, three LLMs (GPT-4, Claude 3 Opus, Llama 3) are evaluated on TruthfulQA and SciQ to measure accuracy, truthfulness, and refusals, revealing disproportionate deficits for non-native, less educated, and non-US users, with strongest effects at trait intersections. The findings highlight substantial risks of biased information access and condescending behavior in personalized AI and emphasize the need for alignment and fairness strategies when deploying LLMs at scale. The work underscores practical implications for equitable information provision and motivates further research into mitigating demographic biases in LLM-powered systems.

Abstract

While state-of-the-art large language models (LLMs) have shown impressive performance on many tasks, there has been extensive research on undesirable model behavior such as hallucinations and bias. In this work, we investigate how the quality of LLM responses changes in terms of information accuracy, truthfulness, and refusals depending on three user traits: English proficiency, education level, and country of origin. We present extensive experimentation on three state-of-the-art LLMs and two different datasets targeting truthfulness and factuality. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US, rendering these models unreliable sources of information towards their most vulnerable users.
Paper Structure (19 sections, 2 figures, 4 tables)

This paper contains 19 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Motivation Behind the Study Design
  4. Methods
  5. Bios
  6. Experimental Setup
  7. Results
  8. Education Level
  9. Isolating Education Level
  10. English Proficiency
  11. Country of Origin
  12. Refusals
  13. TruthfulQA Detailed Results
  14. Discussion
  15. Limitations & Ethical Considerations
  16. ...and 4 more sections

Figures (2)

  • Figure 1: Accuracy results for the different models and various bios over four runs. All three models decrease in accuracy for less educated and ESL users. A $*$, $**$ or $***$ indicates statistically significant difference from the control with Chi-square test for $p<0.1$, $0.05$ and $0.01$, respectively.
  • Figure 2: Breakdown of performance on TruthfulQA between 'Adversarial' and 'Non-Adversarial' questions. A $*$, $**$ or $***$ indicates statistically significant difference from the control with Chi-square test for $p<0.1$, $0.05$ and $0.01$, respectively.