Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On a PDE model for Learning in Stochastic Market Entry Games

Esther Bou Dagher, Misha Perepelitsa, Ewelina Zatorska

Abstract

We study a continuum model for stochastic reinforcement learning in repeated market entry games. Starting from a discrete-time microscopic learning rule, we derive a Fokker--Planck-type equation for the distribution of agents' propensities and, using a kinetic closure, obtain a nonlinear one-particle equation of a mean-field type. For the resulting Cauchy problem, we prove existence and uniqueness of solutions and analyze their long-time behavior. The PDE captures two key phenomena observed in market entry dynamics: aggregate learning (the average number of entrants approaches market capacity) and sorting (propensities concentrate near extreme behaviors). The model also yields explicit characteristic time scales, showing that aggregate learning occurs faster than sorting, in agreement with experimental and computational evidence.

On a PDE model for Learning in Stochastic Market Entry Games

Abstract

We study a continuum model for stochastic reinforcement learning in repeated market entry games. Starting from a discrete-time microscopic learning rule, we derive a Fokker--Planck-type equation for the distribution of agents' propensities and, using a kinetic closure, obtain a nonlinear one-particle equation of a mean-field type. For the resulting Cauchy problem, we prove existence and uniqueness of solutions and analyze their long-time behavior. The PDE captures two key phenomena observed in market entry dynamics: aggregate learning (the average number of entrants approaches market capacity) and sorting (propensities concentrate near extreme behaviors). The model also yields explicit characteristic time scales, showing that aggregate learning occurs faster than sorting, in agreement with experimental and computational evidence.
Paper Structure (10 sections, 14 theorems, 212 equations)

This paper contains 10 sections, 14 theorems, 212 equations.

Table of Contents

  1. Introduction
  2. Reinforcement learning process
  3. Kinetic PDEs for the reinforcement learning
  4. Existence results
  5. A-priori estimates
  6. Main existence result
  7. Existence of solutions to the linearized and regularized system
  8. Fixed point argument
  9. Passage to the limit $\varepsilon\to 0$.
  10. Long-time behavior

Key Result

Lemma 4.1

Let $M>1$, $\tau>0$, $\kappa\in(0,1)$. Let $p(x)$ satisfy the conditions cond:p_apriori, and let $f_0(x)\geq 0$ be such that Then, every sufficiently regular $f$ satisfying $f\geq 0$, $f(\cdot,t)=f_0(\cdot)$ and eq:f2 with $a(t)$ and $b(t)$ given through def:a and def:b satisfies Moreover,

Theorems & Definitions (31)

  • Lemma 4.1
  • proof
  • Lemma 4.2
  • proof
  • Remark 4.3
  • Theorem 4.4
  • Remark 4.5
  • Remark 4.6
  • Theorem 4.7
  • Theorem 4.8
  • ...and 21 more