Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs with Personalities in Multi-issue Negotiation Games

Sean Noh, Ho-Chun Herbert Chang

TL;DR

Simulations reveal increase in domain complexity with asymmetric issue valuations improve agreement rates but decrease surplus from aggressive negotiation, indicating that LLMs may have built-in guardrails that default to fair behavior, but can be "jail broken" to exploit agreeable opponents.

Abstract

Powered by large language models (LLMs), AI agents have become capable of many human tasks. Using the most canonical definitions of the Big Five personality, we measure the ability of LLMs to negotiate within a game-theoretical framework, as well as methodological challenges to measuring notions of fairness and risk. Simulations (n=1,500) for both single-issue and multi-issue negotiation reveal increase in domain complexity with asymmetric issue valuations improve agreement rates but decrease surplus from aggressive negotiation. Through gradient-boosted regression and Shapley explainers, we find high openness, conscientiousness, and neuroticism are associated with fair tendencies; low agreeableness and low openness are associated with rational tendencies. Low conscientiousness is associated with high toxicity. These results indicate that LLMs may have built-in guardrails that default to fair behavior, but can be "jail broken" to exploit agreeable opponents. We also offer pragmatic insight in how negotiation bots can be designed, and a framework of assessing negotiation behavior based on game theory and computational social science.

LLMs with Personalities in Multi-issue Negotiation Games

TL;DR

Simulations reveal increase in domain complexity with asymmetric issue valuations improve agreement rates but decrease surplus from aggressive negotiation, indicating that LLMs may have built-in guardrails that default to fair behavior, but can be "jail broken" to exploit agreeable opponents.

Abstract

Powered by large language models (LLMs), AI agents have become capable of many human tasks. Using the most canonical definitions of the Big Five personality, we measure the ability of LLMs to negotiate within a game-theoretical framework, as well as methodological challenges to measuring notions of fairness and risk. Simulations (n=1,500) for both single-issue and multi-issue negotiation reveal increase in domain complexity with asymmetric issue valuations improve agreement rates but decrease surplus from aggressive negotiation. Through gradient-boosted regression and Shapley explainers, we find high openness, conscientiousness, and neuroticism are associated with fair tendencies; low agreeableness and low openness are associated with rational tendencies. Low conscientiousness is associated with high toxicity. These results indicate that LLMs may have built-in guardrails that default to fair behavior, but can be "jail broken" to exploit agreeable opponents. We also offer pragmatic insight in how negotiation bots can be designed, and a framework of assessing negotiation behavior based on game theory and computational social science.
Paper Structure (16 sections, 7 figures)

This paper contains 16 sections, 7 figures.

Table of Contents

  1. Introduction
  2. Results
  3. Discussion
  4. Methods
  5. Appendix

Figures (7)

  • Figure 1: Normalized payoffs of personality-based agents in single and multi issue games (a) including and (b) excluding games ending in default.
  • Figure 2: (a) Toxicity in single-issue bargaining games by personality and (b) payoff advantage for P1 in multi-issue games.
  • Figure 3: Payoffs in (a) single-issue bargaining games by head-to-head match up and (b) multi-issue bargaining games between Hi-Agree and Lo-Agree personalities, excluding games ending in default.
  • Figure 4: (a) SHAP feature analysis on multi-issue games and (b) correlation scatter plot for model prediction ($r^2$ =0.924).
  • Figure 5: (a) SHAP feature analysis on multi-issue games ending in agreement (or excluding games ending in default) and (b) correlation scatter plot for model prediction ($r^2$ =0.937).
  • ...and 2 more figures