Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation

TL;DR

This paper surveys ethical considerations in crowdsourced ML dataset annotation, focusing on how annotator identities and platform relations shape ground truth. It analyzes how socio-cultural backgrounds, lived experiences, compensation, and power dynamics can bias or enrich labels and thereby influence downstream models. The authors propose a concrete set of recommendations across the data pipeline—task formulation, annotator selection, platform/infrastructure, dataset analysis, and documentation/release. They advocate treating annotation disagreement as an informative signal, documenting demographic and experiential factors, and ensuring mechanisms to address worker power imbalances.

Abstract

Human annotations play a crucial role in machine learning (ML) research and development. However, the ethical considerations around the processes and decisions that go into building ML datasets has not received nearly enough attention. In this paper, we survey an array of literature that provides insights into ethical considerations around crowdsourced dataset annotation. We synthesize these insights, and lay out the challenges in this space along two layers: (1) who the annotator is, and how the annotators' lived experiences can impact their annotations, and (2) the relationship between the annotators and the crowdsourcing platforms and what that relationship affords them. Finally, we put forth a concrete set of recommendations and considerations for dataset developers at various stages of the ML data pipeline: task formulation, selection of annotators, platform and infrastructure choices, dataset analysis and evaluation, and dataset documentation and release.