Are generative AI text annotations systematically biased?

TL;DR

GLLMs perform adequate in terms of F1 scores, but differ from manual annotations in terms of prevalence, yield substantively different downstream results, and display systematic bias in that they overlap more with each other than with manual annotations.

Abstract

This paper investigates bias in GLLM annotations by conceptually replicating manual annotations of Boukes (2024). Using various GLLMs (Llama3.1:8b, Llama3.3:70b, GPT4o, Qwen2.5:72b) in combination with five different prompts for five concepts (political content, interactivity, rationality, incivility, and ideology). We find GLLMs perform adequate in terms of F1 scores, but differ from manual annotations in terms of prevalence, yield substantively different downstream results, and display systematic bias in that they overlap more with each other than with manual annotations. Differences in F1 scores fail to account for the degree of bias.

Figures (6)

• Figure 1: Estimated prevalence of rationality according to GLLM annotators compared to manual prevalence, with 95% confidence intervals.
• Figure 2: Estimated correlation of GLLM annotated rationality and video Genre versus manual annotated rationality.
• Figure 3: The number of GLLM annotators that agree with the manual annotations for what share of the YT-replies.
• Figure 4: Bias in terms of normalized correlation coefficient difference between GLLM and manual annotation versus macro average $F_1$.
• Figure 5: Bias in terms of normalized correlation coefficient difference between GLLM and manual annotation versus macro average $F_1$ and positive class $F_1$ for rationality.