Unbending strategies shepherd cooperation and suppress extortion in spatial populations

Abstract

Evolutionary game dynamics on networks typically consider the competition among simple strategies such as cooperation and defection in the Prisoner's Dilemma and summarize the effect of population structure as network reciprocity. However, it remains largely unknown regarding the evolutionary dynamics involving multiple powerful strategies typically considered in repeated games, such as the zero-determinant (ZD) strategies that are able to enforce a linear payoff relationship between them and their co-players. Here, we consider the evolutionary dynamics of always cooperate (AllC), extortionate ZD (extortioners), and unbending players in lattice populations based on the commonly used death-birth updating. Out of the class of unbending strategies, we consider a particular candidate, PSO Gambler, a machine-learning-optimized memory-one strategy, which can foster reciprocal cooperation and fairness among extortionate players. We derive analytical results under weak selection and rare mutations, including pairwise fixation probabilities and long-term frequencies of strategies. In the absence of the third unbending type, extortioners can achieve a half-half split in equilibrium with unconditional cooperators for sufficiently large extortion factors. However, the presence of unbending players fundamentally changes the dynamics and tilts the system to favor unbending cooperation. Most surprisingly, extortioners cannot dominate at all regardless of how large their extortion factor is, and the long-term frequency of unbending players is maintained almost as a constant. Our analytical method is applicable to studying the evolutionary dynamics of multiple strategies in structured populations. Our work provides insights into the interplay between network reciprocity and direct reciprocity, revealing the role of unbending strategies in enforcing fairness and suppressing extortion.

Figures (6)

• Figure 1: Spatiotemporal coevolutionary dynamics of AllC (always cooperate), extortionate zero-determinant strategy (ZD extortioner), and unbending strategy (PSO Gambler) in repeated games. The sequential series of spatial snapshots shows the invasion of a spatial population by mutations (indicated by solid arrows) and competition dynamics under the death-birth update rule (indicated by dashed arrows). The snapshots are taken from a stochastic simulation on a square lattice of size $10\times 10$ with von Neumann neighborhood $k = 4$, selection strength $\beta=0.001$, mutation rate $\mu=0.005$, conventional payoff parameters $R = 3$, $S = 0$, $T = 5$, $P =1$, $\chi =1$ for the zero-determinant strategy, and $[q_1, q_2, q_3, q_4] = [1, 0.52173487, 0, 0.12050939]$ for the memory-one PSO Gambler.
• Figure 2: Time evolution of spatial competition dynamics among three strategies in repeated games: AllC, extortionate ZD, and PSO Gambler under rare mutations. Two different extortion factors for the extortionate ZD strategy are considered: (a) $\chi = 1$ which leads the extortioner to a Tit-for-Tat player, and (b) $\chi = 4$ which enables the extortioner X to unilaterally enforce an unfair payoff relation against its co-player Y as $s_X - P = \chi (s_Y - P)$. As expected, the simulation from (a) indicates neutral drift with equal frequencies of the three strategies. In contrast, the simulation from (b) suggests that the unbending strategy PSO Gambler is most abundant, and its presence can suppress unfair extortion with $\chi > 1$ and promote cooperation in spatial populations. Simulation parameters are as in Fig. 1, except that we also consider $\chi = 4$ for the extortionate ZD strategy.
• Figure 3: Pairwise competition dynamics under rare mutations. We show fixation probabilities $\rho_{ij}$ between the three strategies considered in repeated games: AllC, extortionate ZD (ZD extortioner) with (a) $\chi = 1$ and (b) $\chi = 4$, and unbending strategy (PSO Gambler), labeled as $i = 1, 2, 3$, respectively. In panel (a), for $\chi = 1$, the game dynamics are neutral among all three strategies, and therefore the fixation probability is $\rho_{ij} = 1/N$ where $N$ is the population size. In panel (b), whereas for $\chi = 4$, the game dynamics remain neutral between AllD and PSO but no longer for AllC vs ZD or ZD vs PSO. It is noteworthy that spatial structure can help AllC to invade extortionate ZD ($\rho_{21} > 1/N > \rho_{12}$) as opposed to well-mixed populations; furthermore, compared to AllC, PSO is less likely to be invaded by ZD and more likely to take over ZD. Together, extortion can be suppressed by spatial structure and the presence of unbending strategies such as the PSO Gambler. Simulation parameters are as in Fig. 1.
• Figure 4: Long-term frequencies of the three strategies -- AllC, extortionate ZD (extortioner), and unbending strategy (PSO Gambler) -- in lattice populations. We find good agreement between the simulation results (symbols) and analytical predictions (lines), shown as a function of the extortion factor $\chi$. Notably, the frequency of PSO Gambler remains almost unchanged while the equilibrium frequency of extortioners increases at the expense of AllC. In reference to the competition of AllC vs ZD in spatial populations (dashed lines), ZD can achieve a half-half split in equilibrium with AllC for sufficiently large extortion factors, despite the role of spatial structure in favoring AllC. However, the presence of the unbending strategy, PSO Gambler, fundamentally affects the underlying pairwise competition dynamics (as depicted in Fig. 3), thereby suppressing extortion regardless of how large $\chi$ is. Simulation parameters are as in Fig. 1., except that we vary the extortion factor $\chi$.
• Figure 5: Fixation probabilities between the three strategies considered in repeated games: AllC, extortionate zero-determinant strategy (ZD extortioner) with $\chi \geq 1$, and unbending strategy (PSO Gambler). Simulation parameters are as in Fig. 1, except that we vary the extortion factor $\chi$ for the extortionate ZD strategy.