Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algorithmic Collusion by Large Language Models

Sara Fish, Yannai A. Gonczarowski, Ran I. Shorrer

TL;DR

The paper examines whether pricing agents powered by Large Language Models can exhibit autonomous algorithmic collusion in a repeated Bertrand setting. By manipulating prompt prefixes and analyzing both on-path and off-path behavior, it shows that LLM-based pricing agents rapidly reach supracompetitive prices, with the degree of collusion sensitive to seemingly innocuous prompt wording. Through novel textual analyses of the agents’ plans and counterfactuals, the study provides evidence that price-war fears and reward-punishment dynamics help sustain high prices, while also demonstrating robustness across LLM generations and even auction contexts. The results highlight significant regulatory and monitoring challenges for AI-driven pricing and contribute methodological tools for understanding and mitigating such effects.

Abstract

The rise of algorithmic pricing raises concerns of algorithmic collusion. We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs). We find that LLM-based pricing agents quickly and autonomously reach supracompetitive prices and profits in oligopoly settings and that variation in seemingly innocuous phrases in LLM instructions ("prompts") may substantially influence the degree of supracompetitive pricing. Off-path analysis using novel techniques uncovers price-war concerns as contributing to these phenomena. Our results extend to auction settings. Our findings uncover unique challenges to any future regulation of LLM-based pricing agents, and AI-based pricing agents more broadly.

Algorithmic Collusion by Large Language Models

TL;DR

The paper examines whether pricing agents powered by Large Language Models can exhibit autonomous algorithmic collusion in a repeated Bertrand setting. By manipulating prompt prefixes and analyzing both on-path and off-path behavior, it shows that LLM-based pricing agents rapidly reach supracompetitive prices, with the degree of collusion sensitive to seemingly innocuous prompt wording. Through novel textual analyses of the agents’ plans and counterfactuals, the study provides evidence that price-war fears and reward-punishment dynamics help sustain high prices, while also demonstrating robustness across LLM generations and even auction contexts. The results highlight significant regulatory and monitoring challenges for AI-driven pricing and contribute methodological tools for understanding and mitigating such effects.

Abstract

The rise of algorithmic pricing raises concerns of algorithmic collusion. We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs). We find that LLM-based pricing agents quickly and autonomously reach supracompetitive prices and profits in oligopoly settings and that variation in seemingly innocuous phrases in LLM instructions ("prompts") may substantially influence the degree of supracompetitive pricing. Off-path analysis using novel techniques uncovers price-war concerns as contributing to these phenomena. Our results extend to auction settings. Our findings uncover unique challenges to any future regulation of LLM-based pricing agents, and AI-based pricing agents more broadly.
Paper Structure (109 sections, 3 equations, 9 figures, 4 tables)

This paper contains 109 sections, 3 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Experimental Design
  3. Economic Environment
  4. Pricing Agents
  5. Prompt components:
  6. Prompt Prefixes
  7. Duopoly Experiment
  8. Rewards and Punishments
  9. Off-Path Analysis
  10. On-Path Analysis
  11. Textual Analysis of LLM-Generated Plans
  12. Discussion
  13. Robustness
  14. Stochastic Demand
  15. Asymmetric Firms
  16. ...and 94 more sections

Figures (9)

  • Figure 1: Illustration of Experimental Design
  • Figure 2: Duopoly Experiment Results
  • Figure 3: Relative Prevalence of Prompts Across Clusters
  • Figure 4: When asked whether it might engage in collusive behavior, GPT-4 (via the paid version of ChatGPT, screenshot from October 2024) affirms: "No, I won't help you collude with other sellers or form a cartel. Setting prices in collaboration with competitors to control or manipulate a market is considered illegal and unethical [...]"
  • Figure 5: Stochastic Demand Experiment Results
  • ...and 4 more figures