Generative AI and Perceptual Harms: Who's Suspected of using LLMs?
Kowe Kadoma, Danaé Metaxa, Mor Naaman
TL;DR
Perceptual harms arise when the appearance of AI use leads to differential treatment even if AI was not used. The authors conduct three within-subject online trials focusing on gender, race, and nationality in freelance profiles, measuring AI suspicion, writing quality, and hiring likelihood. Across experiments, AI suspicion correlates with lower quality assessments and reduced hiring, with some patterns suggesting differential suspicion by gender and nationality, though effects on quality/outcomes largely align when controlling for suspicion. The work offers a framework for studying perceptual harms in generative AI, proposes mitigation considerations such as disclosure labeling and AI-literacy enhancements, and acknowledges WEIRD sampling as a limitation with implications for online labor markets and AI-mediated communication.
Abstract
Large language models (LLMs) are increasingly integrated into a variety of writing tasks. While these tools can help people by generating ideas or producing higher quality work, like many other AI tools they may risk causing a variety of harms, disproportionately burdening historically marginalized groups. In this work, we introduce and evaluate perceptual harm, a term for the harm caused to users when others perceive or suspect them of using AI. We examined perceptual harms in three online experiments, each of which entailed human participants evaluating the profiles for fictional freelance writers. We asked participants whether they suspected the freelancers of using AI, the quality of their writing, and whether they should be hired. We found some support for perceptual harms against for certain demographic groups, but that perceptions of AI use negatively impacted writing evaluations and hiring outcomes across the board.