Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Design With Large Language Models

Paul Duetting, Safwan Hossain, Tao Lin, Renato Paes Leme, Sai Srivatsa Ravindranath, Haifeng Xu, Song Zuo

TL;DR

This paper formalizes a language-based notion of framing and bridges this to the popular Bayesian-persuasion model, and forms a theoretical model based on access to a framing-to-belief oracle that enables us to precisely characterize when solely optimizing framing or jointly optimizing framing and signaling is tractable.

Abstract

Information design is typically studied through the lens of Bayesian signaling, where signals shape beliefs based on their correlation with the true state of the world. However, Behavioral Economics and Psychology emphasize that human decision-making is more complex and can depend on how information is framed. This paper formalizes a language-based notion of framing and bridges this to the popular Bayesian-persuasion model. We model framing as a possibly non-Bayesian, linguistic way to influence a receiver's belief, while a signaling (or recommendation) scheme can further refine this belief in the classic Bayesian way. A key challenge in systematically optimizing in this framework is the vast space of possible framings and the difficulty of predicting their effects on receivers. Based on growing evidence that Large Language Models (LLMs) can effectively serve as proxies for human behavior, we formulate a theoretical model based on access to a framing-to-belief oracle. This model then enables us to precisely characterize when solely optimizing framing or jointly optimizing framing and signaling is tractable. We substantiate our theoretical analysis with an empirical algorithm that leverages LLMs to (1) approximate the framing-to-belief oracle, and (2) optimize over language space using a hill-climbing method. We apply this to two marketing-inspired case studies and validate the effectiveness through analytical and human evaluation.

Information Design With Large Language Models

TL;DR

This paper formalizes a language-based notion of framing and bridges this to the popular Bayesian-persuasion model, and forms a theoretical model based on access to a framing-to-belief oracle that enables us to precisely characterize when solely optimizing framing or jointly optimizing framing and signaling is tractable.

Abstract

Information design is typically studied through the lens of Bayesian signaling, where signals shape beliefs based on their correlation with the true state of the world. However, Behavioral Economics and Psychology emphasize that human decision-making is more complex and can depend on how information is framed. This paper formalizes a language-based notion of framing and bridges this to the popular Bayesian-persuasion model. We model framing as a possibly non-Bayesian, linguistic way to influence a receiver's belief, while a signaling (or recommendation) scheme can further refine this belief in the classic Bayesian way. A key challenge in systematically optimizing in this framework is the vast space of possible framings and the difficulty of predicting their effects on receivers. Based on growing evidence that Large Language Models (LLMs) can effectively serve as proxies for human behavior, we formulate a theoretical model based on access to a framing-to-belief oracle. This model then enables us to precisely characterize when solely optimizing framing or jointly optimizing framing and signaling is tractable. We substantiate our theoretical analysis with an empirical algorithm that leverages LLMs to (1) approximate the framing-to-belief oracle, and (2) optimize over language space using a hill-climbing method. We apply this to two marketing-inspired case studies and validate the effectiveness through analytical and human evaluation.

Paper Structure

This paper contains 45 sections, 15 theorems, 60 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Our Model.
  4. Problem Variants.
  5. Theoretical Results.
  6. Empirical Results.
  7. Broader Implications.
  8. Additional Related Work
  9. Algorithmic information design.
  10. Information design with LLMs.
  11. Additional evidence for framing effects.
  12. LLMs as proxys for human behavior.
  13. Model
  14. Bayesian Persuasion and Signaling.
  15. Framing.
  16. ...and 30 more sections

Key Result

Proposition 1

For any instance $\mathcal{I}$ with a given signaling scheme $\pi$, the optimal sender utility can always be achieved by some deterministic framing $c^*$.

Figures (4)

  • Figure 1: Diagram of our proposed framework for optimizing framing and signaling. It includes LLMs searching the framing space, verifying it for correctness, and generating framing-induced beliefs. It also includes poly-time analytical solvers to compute optimal signaling for a given belief.
  • Figure 2: Means (across 50 runs) from both LLM and human-generated beliefs with 90% confidence intervals on the Henry instance.
  • Figure 3: Means (across 50 runs) from both LLM and human-generated beliefs with 90% confidence intervals on the Lilly instance.
  • Figure 4: Scores over multiple iterations of the LLM generating framing. Final score is the product of the utility and the correctness score.

Theorems & Definitions (36)

  • Definition 1: Framing Space
  • Definition 2: Information Design with Framing
  • Definition 3: Equilibrium
  • Proposition 1
  • Corollary 1
  • Definition 4: Bayesian Stackelberg game
  • Proposition 2
  • proof
  • Theorem 1: NP-hardness
  • proof : Proof Sketch
  • ...and 26 more