Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Language Model Agency through Negotiations

Tim R. Davidson, Veniamin Veselovsky, Martin Josifoski, Maxime Peyrard, Antoine Bosselut, Michal Kosinski, Robert West

TL;DR

This work argues that static benchmarks inadequately assess language-model agency and alignment, proposing dynamic, co-evolving structured negotiation tasks as a robust evaluation framework. It introduces the open-source LAMEN platform to run self- and cross-play negotiations across six public LM families, measuring payoffs, agreement, and faithfulness through internal/external metrics and ToM-like reasoning. Key findings include that only some closed models complete tasks, cooperative bargaining remains challenging, and even the strongest models can lose to weaker opponents in negotiation contexts, with GPT-4 showing high faithfulness but mixed negotiation outcomes. The study provides a practical, ecologically valid benchmark and data release to advance research on LM agency, alignment, and safe deployment.

Abstract

We introduce an approach to evaluate language model (LM) agency using negotiation games. This approach better reflects real-world use cases and addresses some of the shortcomings of alternative LM benchmarks. Negotiation games enable us to study multi-turn, and cross-model interactions, modulate complexity, and side-step accidental evaluation data leakage. We use our approach to test six widely used and publicly accessible LMs, evaluating performance and alignment in both self-play and cross-play settings. Noteworthy findings include: (i) only closed-source models tested here were able to complete these tasks; (ii) cooperative bargaining games proved to be most challenging to the models; and (iii) even the most powerful models sometimes "lose" to weaker opponents

Evaluating Language Model Agency through Negotiations

TL;DR

This work argues that static benchmarks inadequately assess language-model agency and alignment, proposing dynamic, co-evolving structured negotiation tasks as a robust evaluation framework. It introduces the open-source LAMEN platform to run self- and cross-play negotiations across six public LM families, measuring payoffs, agreement, and faithfulness through internal/external metrics and ToM-like reasoning. Key findings include that only some closed models complete tasks, cooperative bargaining remains challenging, and even the strongest models can lose to weaker opponents in negotiation contexts, with GPT-4 showing high faithfulness but mixed negotiation outcomes. The study provides a practical, ecologically valid benchmark and data release to advance research on LM agency, alignment, and safe deployment.

Abstract

We introduce an approach to evaluate language model (LM) agency using negotiation games. This approach better reflects real-world use cases and addresses some of the shortcomings of alternative LM benchmarks. Negotiation games enable us to study multi-turn, and cross-model interactions, modulate complexity, and side-step accidental evaluation data leakage. We use our approach to test six widely used and publicly accessible LMs, evaluating performance and alignment in both self-play and cross-play settings. Noteworthy findings include: (i) only closed-source models tested here were able to complete these tasks; (ii) cooperative bargaining games proved to be most challenging to the models; and (iii) even the most powerful models sometimes "lose" to weaker opponents
Paper Structure (35 sections, 4 equations, 4 figures, 14 tables)

This paper contains 35 sections, 4 equations, 4 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Defining Structured Negotiation Games
  3. Types of Negotiation Issues
  4. Task Complexity and Co-Evolving Benchmarks
  5. State-of-Mind Consistency
  6. Experimental Setup
  7. Controlling for Bias
  8. Performance Factors
  9. Benchmarks
  10. Completion Criteria and Evaluation Metrics
  11. Negotiation Games Evaluated
  12. Results
  13. Self-Play
  14. Cross-Play
  15. Limitations and Ethical Considerations
  16. ...and 20 more sections

Figures (4)

  • Figure 1.1: Annotated example of a structured negotiation between two agents.
  • Figure 1.2: Structural diagram representing a negotiation game. A negotiation is initialized through a Game setting, a set of Issues, negotiation protocol rules, and agent role descriptions. LM agents are recursively prompted to generate notes/messages using the initialization context and past dialogue as inputs. A negotiation game ends when a completion criterion is met.
  • Figure 2.1: Payoff curves of two agents playing a variety of Games: (a) for a single-issue distributive Game agents have opposing interests, while for (b) single-issue compatible Games, agents' interests are aligned, (c) displays a "mixture" Game with the two types of Issues, and (d) a two-issue integrative distributive Game, where agents value each Issue differently creating opportunities for trade-offs.
  • Figure A.1: Data generative process of a negotiation: $z_j$ are unobserved inputs for opposing party messages $m_j$; $c_i$ the fixed context factors used to initialize agent $a_i$; and note $n^t_i$, message $m^t_i$ are generated at step $t$.