Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP

TL;DR

The paper addresses how annotator cognitive biases in crowdsourced NLP datasets can propagate or amplify model biases. It surveys current bias definitions, debiasing strategies, and the harms of bias, arguing that absolute fairness is infeasible but harms can be minimized, especially when accounting for cultural contexts. It highlights the central role of annotation design and human-in-the-loop processes, proposing an HCI-informed framework to improve transparency, reliability, and fairness in crowdwork. The work offers practical guidelines for diverse recruitment, rigorous calibration, and robust evaluation to reduce bias and enhance accountability in AI systems relying on human-labeled data.

Abstract

With the rapid proliferation of artificial intelligence, there is growing concern over its potential to exacerbate existing biases and societal disparities and introduce novel ones. This issue has prompted widespread attention from academia, policymakers, industry, and civil society. While evidence suggests that integrating human perspectives can mitigate bias-related issues in AI systems, it also introduces challenges associated with cognitive biases inherent in human decision-making. Our research focuses on reviewing existing methodologies and ongoing investigations aimed at understanding annotation attributes that contribute to bias.