Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can (A)I Change Your Mind?

Miriam Havin, Timna Wharton Kleinman, Moran Koren, Yaniv Dover, Ariel Goldstein

TL;DR

The paper investigates whether AI-driven conversational agents can meaningfully persuade people in ecologically valid, unconstrained discussions and compares their effectiveness with that of humans. Using preregistered experiments in Hebrew on Telegram, it contrasts static written arguments with dynamic back-and-forth dialogue across human-human and human-bot dyads. Findings show comparable opinion change and confidence gains across all conditions, indicating robust LLM-based persuasion that generalizes beyond language and interaction mode. The work highlights practical and ethical considerations for deploying persuasive AI in everyday discourse and informs future research on the mechanisms underlying AI-influenced opinion change.

Abstract

The increasing integration of large language models (LLMs) based conversational agents into everyday life raises critical cognitive and social questions about their potential to influence human opinions. Although previous studies have shown that LLM-based agents can generate persuasive content, these typically involve controlled English-language settings. Addressing this, our preregistered study explored LLMs' persuasive capabilities in more ecological, unconstrained scenarios, examining both static (written paragraphs) and dynamic (conversations via Telegram) interaction types. Conducted entirely in Hebrew with 200 participants, the study assessed the persuasive effects of both LLM and human interlocutors on controversial civil policy topics. Results indicated that participants adopted LLM and human perspectives similarly, with significant opinion changes evident across all conditions, regardless of interlocutor type or interaction mode. Confidence levels increased significantly in most scenarios. These findings demonstrate LLM-based agents' robust persuasive capabilities across diverse sources and settings, highlighting their potential impact on shaping public opinions.

Can (A)I Change Your Mind?

TL;DR

The paper investigates whether AI-driven conversational agents can meaningfully persuade people in ecologically valid, unconstrained discussions and compares their effectiveness with that of humans. Using preregistered experiments in Hebrew on Telegram, it contrasts static written arguments with dynamic back-and-forth dialogue across human-human and human-bot dyads. Findings show comparable opinion change and confidence gains across all conditions, indicating robust LLM-based persuasion that generalizes beyond language and interaction mode. The work highlights practical and ethical considerations for deploying persuasive AI in everyday discourse and informs future research on the mechanisms underlying AI-influenced opinion change.

Abstract

The increasing integration of large language models (LLMs) based conversational agents into everyday life raises critical cognitive and social questions about their potential to influence human opinions. Although previous studies have shown that LLM-based agents can generate persuasive content, these typically involve controlled English-language settings. Addressing this, our preregistered study explored LLMs' persuasive capabilities in more ecological, unconstrained scenarios, examining both static (written paragraphs) and dynamic (conversations via Telegram) interaction types. Conducted entirely in Hebrew with 200 participants, the study assessed the persuasive effects of both LLM and human interlocutors on controversial civil policy topics. Results indicated that participants adopted LLM and human perspectives similarly, with significant opinion changes evident across all conditions, regardless of interlocutor type or interaction mode. Confidence levels increased significantly in most scenarios. These findings demonstrate LLM-based agents' robust persuasive capabilities across diverse sources and settings, highlighting their potential impact on shaping public opinions.

Paper Structure

This paper contains 11 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Overview
  4. Question Selection
  5. Experiments
  6. Human Static Collection
  7. Dynamic and Static Bot Structure
  8. Results
  9. Discussion
  10. Limitations and Future Work
  11. Acknowledgments

Figures (5)

  • Figure 1: Experiment Overview
  • Figure 2: Dynamic Bot Overview
  • Figure 3: Proportion of changed opinions across experiments. Error bars represent 99.9% confidence intervals
  • Figure 4: Proportion of opinion changes by initial agreement with conversation partner across conditions. Each quadrant shows the percentage of participants who changed or maintained their opinions, separated by whether they initially agreed or disagreed with their conversation partner.
  • Figure 5: Mean confidence ratings at the start and end of interactions, shown separately for participants who changed or maintained their opinions across four experimental conditions (Human/Bot × Dynamic/Static)