Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Paying to Do Better: Games with Payments between Learning Agents

Yoav Kolumbus, Joe Halpern, Éva Tardos

TL;DR

This work investigates how monetary transfers between learning agents affect long-run outcomes in repeated games, with a focus on auctions. The authors introduce a payment-policy game in which players pre-commit to payment schemes and then deploy no-regret learners who act in the underlying game; they analyze both second-price and first-price auctions, showing that equilibria under payments can yield near-fully cooperative welfare for bidders while eroding auction revenue. The results extend to broader finite games, demonstrating that unilateral or bilateral payments can steer dynamics toward optimal welfare or Stackelberg-like outcomes, often at the expense of the mechanism designer. The findings reveal a new dimension in mechanism design for AI-driven markets, highlighting potential efficiency gains alongside risks of covert collusion, and underscore the need for robust design principles that account for inter-agent financial interactions.

Abstract

In repeated games, such as auctions, players typically use learning algorithms to choose their actions. The use of such autonomous learning agents has become widespread on online platforms. In this paper, we explore the impact of players incorporating monetary transfer policies into their agents' algorithms, aiming to influence behavior in their favor through the dynamics between the agents. Our focus is on understanding when players have incentives to make use of monetary transfers, how such payments may affect learning dynamics, and what the implications are for welfare and its distribution among the players. We propose a simple and general game-theoretic model to capture such scenarios. Our results on general games show that in a very broad class of games, self-interested players benefit from letting their learning agents make payments to other learners during the game dynamics, and that in many cases, this kind of behavior improves welfare for all players. Our results on first- and second-price auctions show that in equilibria of the ``payment policy game,'' the agents' dynamics reach strong collusive outcomes with low revenue for the auctioneer. These results raise new questions and highlight a challenge for mechanism design in systems where automated learning agents can benefit from interacting with their peers in the digital ecosystem and outside the boundaries of the mechanism.

Paying to Do Better: Games with Payments between Learning Agents

TL;DR

This work investigates how monetary transfers between learning agents affect long-run outcomes in repeated games, with a focus on auctions. The authors introduce a payment-policy game in which players pre-commit to payment schemes and then deploy no-regret learners who act in the underlying game; they analyze both second-price and first-price auctions, showing that equilibria under payments can yield near-fully cooperative welfare for bidders while eroding auction revenue. The results extend to broader finite games, demonstrating that unilateral or bilateral payments can steer dynamics toward optimal welfare or Stackelberg-like outcomes, often at the expense of the mechanism designer. The findings reveal a new dimension in mechanism design for AI-driven markets, highlighting potential efficiency gains alongside risks of covert collusion, and underscore the need for robust design principles that account for inter-agent financial interactions.

Abstract

In repeated games, such as auctions, players typically use learning algorithms to choose their actions. The use of such autonomous learning agents has become widespread on online platforms. In this paper, we explore the impact of players incorporating monetary transfer policies into their agents' algorithms, aiming to influence behavior in their favor through the dynamics between the agents. Our focus is on understanding when players have incentives to make use of monetary transfers, how such payments may affect learning dynamics, and what the implications are for welfare and its distribution among the players. We propose a simple and general game-theoretic model to capture such scenarios. Our results on general games show that in a very broad class of games, self-interested players benefit from letting their learning agents make payments to other learners during the game dynamics, and that in many cases, this kind of behavior improves welfare for all players. Our results on first- and second-price auctions show that in equilibria of the ``payment policy game,'' the agents' dynamics reach strong collusive outcomes with low revenue for the auctioneer. These results raise new questions and highlight a challenge for mechanism design in systems where automated learning agents can benefit from interacting with their peers in the digital ecosystem and outside the boundaries of the mechanism.
Paper Structure (15 sections, 13 theorems, 17 equations, 2 figures)

This paper contains 15 sections, 13 theorems, 17 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Model
  3. Further Related Work
  4. Auctions with Payments between Learning Agents
  5. Second-price auctions
  6. First-price auctions
  7. Proof of Theorem \ref{['thm:first-price-cooperative-outcomes']}
  8. Manipulations by One Player
  9. Two-Player Games
  10. Conclusion
  11. Prisoner's Dilemma Games
  12. Equilibria of the game from Figure \ref{['fig:PD-game-a']}:
  13. Price of Stability and Price of Anarchy:
  14. Extension of Lemma \ref{['thm:lemma-first-price-mean-based-NE-CDFs']}
  15. Additional Definitions

Key Result

Theorem 1

The single-item second-price auction, where the values of the top two valued players are $v_1 > v_2$, is not stable for any regret-minimizing agents for the players. Furthermore, the payment-policy game of this auction has an $\epsilon$-equilibrium where the players capture the full welfare, and the

Figures (2)

  • Figure 1: A prisoner's dilemma game.
  • Figure 2: Dynamics of Hedge algorithms with payments between agents in first-price auctions.

Theorems & Definitions (40)

  • Definition 1
  • Theorem 1
  • proof
  • Claim 1
  • proof
  • Claim 2
  • proof
  • Theorem 2
  • Remark
  • Lemma 1
  • ...and 30 more