Security Benefits and Side Effects of Labeling AI-Generated Images
Sandra Höltervennhoff, Jonas Ricker, Maike M. Raphael, Charlotte Schwedes, Rebecca Weil, Asja Fischer, Thorsten Holz, Lea Schönherr, Sascha Fahl
TL;DR
This study investigates the security benefits and side effects of labeling AI-generated images through a mixed-methods design combining five focus groups and a preregistered online survey (N=1,354 valid). It finds that while AI labels can reduce belief in misinformation linked to AI-generated content, they can also cause overreliance, reduce trust for true claims, and paradoxically elevate the credibility of some non-AI misinformation; mislabeling risks further erode trust. The results highlight that labels are not a panacea and must be implemented with simple, consistent rules, transparent mechanisms, and robust handling of mislabeling to achieve meaningful security benefits. The findings have important regulatory and platform-design implications for disclosure policies and the broader effort to combat AI-enabled misinformation. Overall, labeling is a valuable transparency tool but should be complemented with other safeguards and carefully managed to avoid unintended trust distortions.
Abstract
Generative artificial intelligence is developing rapidly, impacting humans' interaction with information and digital media. It is increasingly used to create deceptively realistic misinformation, so lawmakers have imposed regulations requiring the disclosure of AI-generated content. However, only little is known about whether these labels reduce the risks of AI-generated misinformation. Our work addresses this research gap. Focusing on AI-generated images, we study the implications of labels, including the possibility of mislabeling. Assuming that simplicity, transparency, and trust are likely to impact the successful adoption of such labels, we first qualitatively explore users' opinions and expectations of AI labeling using five focus groups. Second, we conduct a pre-registered online survey with over 1300 U.S. and EU participants to quantitatively assess the effect of AI labels on users' ability to recognize misinformation containing either human-made or AI-generated images. Our focus groups illustrate that, while participants have concerns about the practical implementation of labeling, they consider it helpful in identifying AI-generated images and avoiding deception. However, considering security benefits, our survey revealed an ambiguous picture, suggesting that users might over-rely on labels. While inaccurate claims supported by labeled AI-generated images were rated less credible than those with unlabeled AI-images, the belief in accurate claims also decreased when accompanied by a labeled AI-generated image. Moreover, we find the undesired side effect that human-made images conveying inaccurate claims were perceived as more credible in the presence of labels.