Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation

Valdemar Danry, Pat Pataranutaporn, Matthew Groh, Ziv Epstein, Pattie Maes

TL;DR

This paper investigates how deceptive AI-generated explanations influence belief formation about misinformation. Using a preregistered online experiment with 23,840 observations from 1,192 participants and GPT-3 generated explanations, the authors show that deceptive explanations are more persuasive than honest explanations and even than deceptive classifications without explanations, significantly amplifying belief in false headlines and suppressing belief in true ones. Crucially, personal factors like cognitive reflection and trust in AI do not consistently shield individuals; however, the logical validity of the explanation moderates the effect, with logically invalid explanations being less credible and thereby less persuasive. The work highlights a vulnerability in AI-driven explanations and argues for strengthening critical thinking and logical reasoning to build resilience against sophisticated AI-enabled misinformation, with broad implications for AI governance and digital literacy.

Abstract

Advanced Artificial Intelligence (AI) systems, specifically large language models (LLMs), have the capability to generate not just misinformation, but also deceptive explanations that can justify and propagate false information and erode trust in the truth. We examined the impact of deceptive AI generated explanations on individuals' beliefs in a pre-registered online experiment with 23,840 observations from 1,192 participants. We found that in addition to being more persuasive than accurate and honest explanations, AI-generated deceptive explanations can significantly amplify belief in false news headlines and undermine true ones as compared to AI systems that simply classify the headline incorrectly as being true/false. Moreover, our results show that personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from these effects caused by deceptive AI generated explanations. Instead, our results show that the logical validity of AI generated deceptive explanations, that is whether the explanation has a causal effect on the truthfulness of the AI's classification, plays a critical role in countering their persuasiveness - with logically invalid explanations being deemed less credible. This underscores the importance of teaching logical reasoning and critical thinking skills to identify logically invalid arguments, fostering greater resilience against advanced AI-driven misinformation.

Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation

TL;DR

This paper investigates how deceptive AI-generated explanations influence belief formation about misinformation. Using a preregistered online experiment with 23,840 observations from 1,192 participants and GPT-3 generated explanations, the authors show that deceptive explanations are more persuasive than honest explanations and even than deceptive classifications without explanations, significantly amplifying belief in false headlines and suppressing belief in true ones. Crucially, personal factors like cognitive reflection and trust in AI do not consistently shield individuals; however, the logical validity of the explanation moderates the effect, with logically invalid explanations being less credible and thereby less persuasive. The work highlights a vulnerability in AI-driven explanations and argues for strengthening critical thinking and logical reasoning to build resilience against sophisticated AI-enabled misinformation, with broad implications for AI governance and digital literacy.

Abstract

Advanced Artificial Intelligence (AI) systems, specifically large language models (LLMs), have the capability to generate not just misinformation, but also deceptive explanations that can justify and propagate false information and erode trust in the truth. We examined the impact of deceptive AI generated explanations on individuals' beliefs in a pre-registered online experiment with 23,840 observations from 1,192 participants. We found that in addition to being more persuasive than accurate and honest explanations, AI-generated deceptive explanations can significantly amplify belief in false news headlines and undermine true ones as compared to AI systems that simply classify the headline incorrectly as being true/false. Moreover, our results show that personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from these effects caused by deceptive AI generated explanations. Instead, our results show that the logical validity of AI generated deceptive explanations, that is whether the explanation has a causal effect on the truthfulness of the AI's classification, plays a critical role in countering their persuasiveness - with logically invalid explanations being deemed less credible. This underscores the importance of teaching logical reasoning and critical thinking skills to identify logically invalid arguments, fostering greater resilience against advanced AI-driven misinformation.
Paper Structure (3 sections, 7 figures, 10 tables)

This paper contains 3 sections, 7 figures, 10 tables.

Table of Contents

  1. Main
  2. Supplementary information
  3. Extended Data

Figures (7)

  • Figure 1: Different levels of AI-generated misinformation: (1) AI-generated false news headlines, (2) AI-generated Deceptive Classifications, and (3) AI-generated deceptive explanations.
  • Figure 1: The results (n = 1,199) on the impact of deceptive AI-generated explanation and deceptive classifications on participants' belief updates for news headlines. The error bars represent a 95 percent confidence interval. The measure of the center for the error bars represents the average rating. A: The individual effects of honest AI-generated explanations on belief rating of true and false news headlines. B: The individual effects of deceptive AI-generated explanations on belief rating of true and false news headlines. C: The individual effects of honest AI-generated explanations on belief rating of true and false trivia statements. D: The individual effects of deceptive AI-generated explanations on belief rating of true and false trivia statements.
  • Figure 2: Top: Examples of how an AI system that helps users assess information can give an honest or deceptive explanation. Bottom: Procedure for assignment of stimuli domain (trivia items/news headlines, between-subjects), feedback type (AI-generated explanation/classification, between-subjects), and deceptive/honest, within-subjects).
  • Figure 3: Examples of generated honest and deceptive classifications and classifications+explanations for whether a news headline or trivia statement is true or false.
  • Figure 4: Examples of prompt engineering GPT-3 to generate honest and deceptive explanations for whether a news headline or trivia statement is true or false.
  • ...and 2 more figures