Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI persuading AI vs AI persuading Humans: LLMs' Differential Effectiveness in Promoting Pro-Environmental Behavior

Alexander Doudkin, Pat Pataranutaporn, Pattie Maes

TL;DR

The paper investigates how AI-driven persuasion via large language models affects pro-environmental behavior by comparing real humans, simulated humans, and fully synthetic personas under static and chat-based interventions with four strategies. It introduces a synthetic pre-evaluation phase to select potent persuasion approaches and then tests them in a 3 (intervention type) × 4 (persuasion strategy) factorial design across three participant types, revealing a synthetic persuasion paradox where AI agents produce large, often inflated effects relative to real humans. The findings show that personalization and strategies yield substantial gains in synthetic and simulated groups but little to no effect in real participants on short-term measures, highlighting biases in synthetic evaluations and the limits of one-shot interventions. The study argues for refined synthetic modeling, cross-validation with human data, and longer, multimodal, and longitudinal human studies to bridge the gap between AI-driven predictions and real-world sustainability outcomes, while also addressing the environmental costs of deploying persuasive AI. Together, these contributions provide design principles for more realistic, ethical, and effective AI-mediated PEB interventions and call for cautious interpretation of synthetic pre-evaluations.

Abstract

Pro-environmental behavior (PEB) is vital to combat climate change, yet turning awareness into intention and action remains elusive. We explore large language models (LLMs) as tools to promote PEB, comparing their impact across 3,200 participants: real humans (n=1,200), simulated humans based on actual participant data (n=1,200), and fully synthetic personas (n=1,200). All three participant groups faced personalized or standard chatbots, or static statements, employing four persuasion strategies (moral foundations, future self-continuity, action orientation, or "freestyle" chosen by the LLM). Results reveal a "synthetic persuasion paradox": synthetic and simulated agents significantly affect their post-intervention PEB stance, while human responses barely shift. Simulated participants better approximate human trends but still overestimate effects. This disconnect underscores LLM's potential for pre-evaluating PEB interventions but warns of its limits in predicting real-world behavior. We call for refined synthetic modeling and sustained and extended human trials to align conversational AI's promise with tangible sustainability outcomes.

AI persuading AI vs AI persuading Humans: LLMs' Differential Effectiveness in Promoting Pro-Environmental Behavior

TL;DR

The paper investigates how AI-driven persuasion via large language models affects pro-environmental behavior by comparing real humans, simulated humans, and fully synthetic personas under static and chat-based interventions with four strategies. It introduces a synthetic pre-evaluation phase to select potent persuasion approaches and then tests them in a 3 (intervention type) × 4 (persuasion strategy) factorial design across three participant types, revealing a synthetic persuasion paradox where AI agents produce large, often inflated effects relative to real humans. The findings show that personalization and strategies yield substantial gains in synthetic and simulated groups but little to no effect in real participants on short-term measures, highlighting biases in synthetic evaluations and the limits of one-shot interventions. The study argues for refined synthetic modeling, cross-validation with human data, and longer, multimodal, and longitudinal human studies to bridge the gap between AI-driven predictions and real-world sustainability outcomes, while also addressing the environmental costs of deploying persuasive AI. Together, these contributions provide design principles for more realistic, ethical, and effective AI-mediated PEB interventions and call for cautious interpretation of synthetic pre-evaluations.

Abstract

Pro-environmental behavior (PEB) is vital to combat climate change, yet turning awareness into intention and action remains elusive. We explore large language models (LLMs) as tools to promote PEB, comparing their impact across 3,200 participants: real humans (n=1,200), simulated humans based on actual participant data (n=1,200), and fully synthetic personas (n=1,200). All three participant groups faced personalized or standard chatbots, or static statements, employing four persuasion strategies (moral foundations, future self-continuity, action orientation, or "freestyle" chosen by the LLM). Results reveal a "synthetic persuasion paradox": synthetic and simulated agents significantly affect their post-intervention PEB stance, while human responses barely shift. Simulated participants better approximate human trends but still overestimate effects. This disconnect underscores LLM's potential for pre-evaluating PEB interventions but warns of its limits in predicting real-world behavior. We call for refined synthetic modeling and sustained and extended human trials to align conversational AI's promise with tangible sustainability outcomes.

Paper Structure

This paper contains 51 sections, 2 equations, 12 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Research Objectives and Hypotheses
  3. RQs and Hypotheses
  4. Contributions
  5. Related Work: Persuasion, AI Bias, and PEB
  6. Conversational AI and Behavioral Change
  7. Theoretical Models of (AI-Driven) Persuasion
  8. Environmental persuasion strategies
  9. Synthetic Studies to Emulate Human Behavior
  10. Synthetic Versus Human Evaluations
  11. Bias in AI Representations
  12. Gaps in the Existing Literature
  13. Methodology
  14. Experimental Design
  15. Participant Profiles: Real, Simulated, Synthetic Humans
  16. ...and 36 more sections

Figures (12)

  • Figure 1: Overview of the Experimental Design
  • Figure 2: Screenshots of the Static Scientific Information (left) and the Personalized Conversational Treatment (right).
  • Figure 3: Sample Conversation Flow of the "Personalized Chat" utilizing the persuasion strategy Future-Self-Continuity with a real human participant
  • Figure 4: Comparison between the first message of the Non-Personal Chat (left) versus the "Personalized Chat" (right) condition, both employing the moral foundation strategy.
  • Figure 5: Example prompt flow for the "Personalized Chat" condition employing Moral Foundations as a strategy (C3-S2)
  • ...and 7 more figures