On the discrete-time origins of the replicator dynamics: From convergence to instability and chaos

TL;DR

Problem: whether discrete-time learning and evolution reproduce the RD limit and under what conditions they converge or become chaotic. Approach: recast three discrete-time models—Model I (bio), Model II (PPI), Model III (EW)—as autonomous maps $x_{n+1}=f_{\\delta}(x_n)$ on the unit interval for a symmetric $2\\times2$ congestion game, and analyze fixed points, stability, and chaotic behavior using Schwarzian derivatives and period-3 arguments. Key contributions: (i) Model I globally converges to the Nash equilibrium for all $\\delta>0$; (ii) Model II exhibits convergence, repulsion, period-doubling, and Li–Yorke chaos for large $\\delta$ depending on $p$; (iii) Model III yields instability of all equilibria and Li–Yorke chaos for large $\\delta$; (iv) the results illustrate that discretization can qualitatively alter long-run outcomes despite identical continuous-time limits. Significance: the work provides a concrete cautionary tale about using the continuous-time RD as a universal predictor across biology, economics, and machine learning, highlighting that discretization profoundly affects long-run dynamics.

Abstract

We consider three distinct discrete-time models of learning and evolution in games: a biological model based on intra-species selective pressure, the dynamics induced by pairwise proportional imitation, and the exponential / multiplicative weights (EW) algorithm for online learning. Even though these models share the same continuous-time limit - the replicator dynamics - we show that second-order effects play a crucial role and may lead to drastically different behaviors in each model, even in very simple, symmetric $2\times2$ games. Specifically, we study the resulting discrete-time dynamics in a class of parametrized congestion games, and we show that (i) in the biological model of intra-species competition, the dynamics remain convergent for any parameter value; (ii) the dynamics of pairwise proportional imitation exhibit an entire range of behaviors for larger time steps and different equilibrium configurations (stability, instability, and even Li-Yorke chaos); while (iii) in the EW algorithm, increasing the time step (almost) inevitably leads to chaos (again, in the formal, Li-Yorke sense). This divergence of behaviors comes in stark contrast to the globally convergent behavior of the replicator dynamics, and serves to delineate the extent to which the replicator dynamics provide a useful predictor for the long-run behavior of their discrete-time origins.

Figures (5)

• Figure 1: Bifurcation diagrams for $\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}}$ and $\cramped{f_{\mathclap{\delta}}^{\mathrm{III}}}$ with the equilibrium $p=0.45$. The first bifurcation on both diagrams is at $\delta_0=2/0.45\cdot 0.55=\frac{22}{9}$. But then the second bifurcation for $\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}}$ is much faster (around $\delta=10.5$ while for $\cramped{f_{\mathclap{\delta}}^{\mathrm{III}}}$ it is around $\delta=25$). It is worth pointing out that period 3 can be detected for $\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}}$ at $\delta=12.5$, while for $\cramped{f_{\mathclap{\delta}}^{\mathrm{III}}}$ it is $\delta\approx 55.5$.
• Figure 2: Period diagrams of the small-period attracting periodic orbits associated with the map $\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}}$ (drawn on the black background). The horizontal axes are $\delta \in [4,16]$ and the vertical axes are the asymmetry of cost $p \in [0,1]$. The colors encode the periods of attracting periodic orbits as follows: period 1 (fixed point, which is Nash equilibrium $p$) = yellow, period 2 = red, period 3 = blue, period 4 = green, period 5 = brown, period 6 = cyan, period 7 = darkgray, period 8 = magenta, and period larger than 8 = white. The picture is generated from the following algorithm: 20000 preliminary iterations are discarded. Then a point is considered periodic of period $n$ if $|(\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}})^n(x)-x|<10^{-10}$ and it is not periodic of any period smaller than $n$. As long as we are in the yellow region we have convergence to Nash equilibrium, once we get out of this region almost all trajectories will never converge to the fixed point.
• Figure 3: Period diagrams of the small-period attracting periodic orbits associated with the map $\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}}$ (drawn on the black background). The horizontal axes are $\delta \in [8,16]$ and the vertical axes are the asymmetry of cost $p \in [1/3,1/2]$. The colors encode the periods of attracting periodic orbits as follows: period 1 (fixed point) = yellow, period 2 = red, period 3 = blue, period 4 = green, period 5 = brown, period 6 = cyan, period 7 = darkgray, period 8 = magenta, and period larger than 8 = white. The picture is generated from the following algorithm: 20000 preliminary iterations are discarded. Then a point is considered periodic of period $n$ if $|(\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}})^n(x)-x|<10^{-10}$ and it is not periodic of any period smaller than $n$. On the picture we also draw the black curves for conditions \ref{['f1b6']} and \ref{['f226']}.
• Figure 4: Map $\cramped{f_{\mathclap{\delta}}^{\mathrm{II}}}$ and its third iterate when $p=0.5$ with $\delta=15$ (left) and $\delta=\delta^*=16$ (right).
• Figure 5: Map $\cramped{f_{\mathclap{\delta}}^{\mathrm{III}}}$ and its third iterate when $p=0.35$ with $\delta=20$ (left) and $\delta=35$ (right).

Theorems & Definitions (11)

• Definition 1: Li-Yorke chaos
• Remark
• Theorem 1
• Lemma 1
• proof
• Proposition 1
• proof
• Lemma 2
• proof
• Proposition 2