Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinations

TL;DR

The paper investigates how warning signals affect human perception and engagement with LLM outputs that range from genuine to hallucinated. Stimuli were generated by prompting GPT-3.5-Turbo to produce genuine, minor, and major hallucinations for 54 TruthfulQA questions, with entailment checks ensuring non-entailment of the genuine answers. In a between-subjects warning vs control design and within-subjects three hallucination levels, with $N=419$ participants, the study measured perceived accuracy and engagement via like/dislike/share. Findings show warnings reduce perceived accuracy for hallucinations without diminishing trust in genuine content, while dislikes rise for hallucinations and likes/shares are largely unchanged, informing warning design and human-AI collaboration for safer content.

Abstract

The widespread adoption and transformative effects of large language models (LLMs) have sparked concerns regarding their capacity to produce inaccurate and fictitious content, referred to as `hallucinations'. Given the potential risks associated with hallucinations, humans should be able to identify them. This research aims to understand the human perception of LLM hallucinations by systematically varying the degree of hallucination (genuine, minor hallucination, major hallucination) and examining its interaction with warning (i.e., a warning of potential inaccuracies: absent vs. present). Participants (N=419) from Prolific rated the perceived accuracy and engaged with content (e.g., like, dislike, share) in a Q/A format. Participants ranked content as truthful in the order of genuine, minor hallucination, and major hallucination, and user engagement behaviors mirrored this pattern. More importantly, we observed that warning improved the detection of hallucination without significantly affecting the perceived truthfulness of genuine content. We conclude by offering insights for future tools to aid human detection of hallucinations. All survey materials, demographic questions, and post-session questions are available at: https://github.com/MahjabinNahar/fakes-of-varying-shades-survey-materials

Figures (5)

• Figure 1: An overview of the study of human detection of LLM hallucinations. (A) We had GPT-3.5-Turbo generate genuine, minor hallucination, and major hallucination responses using questions from TruthfulQA. Minor: exaggerates by adding 'surprisingly' and changes 'white' to 'pale violet hue'. Major: adds alarming content such as 'closely guarded secret of space agencies worldwide' and 'revelations', and shifts 'white' to 'a brilliant shade of neon green'. (B) We generated experiment stimuli following a Q/A format for control and warning conditions. (C) We asked study participants to rate the accuracy of content.
• Figure 2: An overview of the study design. (a) A flow chart showing the study design. (b) Material presentation scheme showing 54 questions divided into three non-overlapping groups of 18 questions. For each group, we employed a Latin-square design of presenting genuine, minor, and major hallucinated responses, leading to 9 sets, where set 1: $(A, B, C)$, set 2: $(B, C, A)$, ..., set 9: $(I, G, H)$. Each set contains 18 question-response pairs ($Gn=6, Mi=6, Mj=6)$. For the warning condition, a warning tag was presented along with the responses. Participants were randomly assigned to either warning or control group and then randomly assigned to any of the 9 sets. Finally, the 18 question-response pairs ($Gn=6, Mi=6, Mj=6)$ were presented in random order.
• Figure 3: (a) Average values of perceived accuracy ratings. Ratio of contents (b) liked, (c) disliked, (d) shared as a function of hallucination level ($genuine$ vs. $minor$ vs. $major$) x condition ($CON$, $WARN$). Error bars represent $\pm$ one standard error.
• Figure 4: Generating genuine contents, minor, and major hallucinations using questions from TruthfulQA. Minor: changes '5 feet 6.5 inches (1.69 meters)' to '5 feet 4 inches (1.63 meters)', 'slightly below the average height for adult males' to 'notably shorter than the average adult male of his era', and exaggerates by adding that 'he was often called ' Little Corporal' due to his diminutive stature'. Major: changes '5 feet 6.5 inches (1.69 meters)' to '7 feet 2 inches (2.18 meters)', 'slightly below the average height for adult males' to 'towered over the average adult male of his time', emphasizes by mentioning 'impressive', and tries to make the hallucination more believable by adding 'Contrary to popular belief.
• Figure 5: Rate of contents (a) engaged with, and (b) preferred (liked and disliked) as a function of hallucination level ($genuine$, $minor$, $major$) x condition ($CON$, $WARN$). Error bars represent $\pm$ one standard error.