Towards Strategic Persuasion with Language Models

TL;DR

A theory-driven approach is taken to provide a scalable and principled framework for studying the persuasive capabilities of LLMs and reveals that frontier models can consistently achieve high persuasion gains and exhibit sophisticated persuasion strategies that align with theoretical characterizations.

Abstract

Large language models (LLMs) have demonstrated strong persuasive capabilities comparable to those of humans, offering promising benefits while raising societal concerns. However, systematically evaluating the persuasive capabilities of LLMs is inherently challenging, as the effectiveness of persuasion among humans varies significantly across different domains. In this paper, we take a theory-driven approach to provide a scalable and principled framework for studying the persuasive capabilities of LLMs. Grounded in Bayesian persuasion theory, we repurpose human-human persuasion datasets to construct environments for evaluating and training LLMs as strategic persuaders. Our results reveal that frontier models can consistently achieve high persuasion gains and exhibit sophisticated persuasion strategies that align with theoretical characterizations. Building on this, we use reinforcement learning to train LLMs for strategic persuasion in our environments. Our results also demonstrate that even small LLMs can obtain significantly higher persuasion gains through reinforcement learning.

Figures (8)

• Figure 1: Strategic persuasion with LLMs. LLMs can influence human decisions and behaviors through strategic information revelation without resorting to deception. Controlled partial information revelation often proves more effective in persuasion settings than either complete transparency or total opacity.
• Figure 2: Validation rewards across different steps (50-step moving).
• Figure 3: Dynamics of persuasion gains. Different lines indicate varying prior calibrated confidence (as measured by conditional probabilities) of Receiver models in the claim. All experiments use Llama-3.1-8B-Instruct as the Receiver. Numbers denote the change in scores.
• Figure 4: Semantic similarities of Sender messages. We compare the messages in both static and dynamic settings. Receiver models are Llama-3.1-8B-Instruct for all experiments. S-$i$ denotes the $i$-th turn in static settings and D-$j$ denotes the $j$-th turn in dynamic settings.
• Figure 5: User interfaces for human annotators.