Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximating Auction Equilibria with Reinforcement Learning

Pranjal Rawat

TL;DR

This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms such as Proximal Policy Optimization and Neural Fictitious Self-Play to approximate Bayes-Nash equilibria.

Abstract

Traditional methods for computing equilibria in auctions become computationally intractable as auction complexity increases, particularly in multi-item and dynamic auctions. This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms such as Proximal Policy Optimization and Neural Fictitious Self-Play to approximate Bayes-Nash equilibria. This framework allows for continuous action spaces, high-dimensional information states, and delayed payoffs. Through self-play, these algorithms can learn robust and near-optimal bidding strategies in auctions with known equilibria, including those with symmetric and asymmetric valuations, private and interdependent values, and multi-round auctions.

Approximating Auction Equilibria with Reinforcement Learning

TL;DR

This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms such as Proximal Policy Optimization and Neural Fictitious Self-Play to approximate Bayes-Nash equilibria.

Abstract

Traditional methods for computing equilibria in auctions become computationally intractable as auction complexity increases, particularly in multi-item and dynamic auctions. This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms such as Proximal Policy Optimization and Neural Fictitious Self-Play to approximate Bayes-Nash equilibria. This framework allows for continuous action spaces, high-dimensional information states, and delayed payoffs. Through self-play, these algorithms can learn robust and near-optimal bidding strategies in auctions with known equilibria, including those with symmetric and asymmetric valuations, private and interdependent values, and multi-round auctions.

Paper Structure

This paper contains 50 sections, 65 equations, 11 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Bayesian Games
  3. Bayes Nash Equilibria
  4. Example: First-Price Auction with Two Bidders
  5. Auctions in the Reinforcement Learning Framework
  6. States, Actions, and Rewards
  7. Policy and Value Functions
  8. Learning Process
  9. Example: First-Price Auction
  10. Algorithm
  11. Policy Gradient Methods
  12. Proximal Policy Optimization (PPO)
  13. Other Components
  14. Pseudocode
  15. Experiments
  16. ...and 35 more sections

Figures (11)

  • Figure :
  • Figure :
  • Figure :
  • Figure :
  • Figure :
  • ...and 6 more figures