Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolutionary Prediction Games

Eden Saig, Nir Rosenfeld

TL;DR

This work introduces evolutionary prediction games, a framework grounded in evolutionary game theory which models such feedback loops as natural-selection processes among groups of users and shows that stable coexistence and mutualistic symbiosis between groups becomes possible under realistic constraints.

Abstract

When a prediction algorithm serves a collection of users, disparities in prediction quality are likely to emerge. If users respond to accurate predictions by increasing engagement, inviting friends, or adopting trends, repeated learning creates a feedback loop that shapes both the model and the population of its users. In this work, we introduce evolutionary prediction games, a framework grounded in evolutionary game theory which models such feedback loops as natural-selection processes among groups of users. Our theoretical analysis reveals a gap between idealized and real-world learning settings: In idealized settings with unlimited data and computational power, repeated learning creates competition and promotes competitive exclusion across a broad class of behavioral dynamics. However, under realistic constraints such as finite data, limited compute, or risk of overfitting, we show that stable coexistence and mutualistic symbiosis between groups becomes possible. We analyze these possibilities in terms of their stability and feasibility, present mechanisms that can sustain their existence, and empirically demonstrate our findings.

Evolutionary Prediction Games

TL;DR

This work introduces evolutionary prediction games, a framework grounded in evolutionary game theory which models such feedback loops as natural-selection processes among groups of users and shows that stable coexistence and mutualistic symbiosis between groups becomes possible under realistic constraints.

Abstract

When a prediction algorithm serves a collection of users, disparities in prediction quality are likely to emerge. If users respond to accurate predictions by increasing engagement, inviting friends, or adopting trends, repeated learning creates a feedback loop that shapes both the model and the population of its users. In this work, we introduce evolutionary prediction games, a framework grounded in evolutionary game theory which models such feedback loops as natural-selection processes among groups of users. Our theoretical analysis reveals a gap between idealized and real-world learning settings: In idealized settings with unlimited data and computational power, repeated learning creates competition and promotes competitive exclusion across a broad class of behavioral dynamics. However, under realistic constraints such as finite data, limited compute, or risk of overfitting, we show that stable coexistence and mutualistic symbiosis between groups becomes possible. We analyze these possibilities in terms of their stability and feasibility, present mechanisms that can sustain their existence, and empirically demonstrate our findings.

Paper Structure

This paper contains 103 sections, 31 theorems, 95 equations, 14 figures.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Evolutionary game theory.
  5. Performativity.
  6. Fairness.
  7. Setting
  8. Population structure.
  9. Dynamics.
  10. Outcomes.
  11. Evolutionary Prediction Games
  12. Accuracy as evolutionary fitness.
  13. Equilibrium.
  14. Induced population dynamics.
  15. Fairness.
  16. ...and 88 more sections

Key Result

Proposition 3.1

Let $F({\bm{p}})$ be an evolutionary prediction game induced by a learning algorithm $\mathcal{A}$, and let ${\bm{p}}^*$ be a Nash equilibrium. Then $h\sim \mathcal{A}(D_{{\bm{p}}^*})$ satisfies overall accuracy equality in expectation.

Figures (14)

  • Figure 1: Natural selection in a two-group setting. Population is a mixture ${\bm{p}}$ of groups; Classifier $h$ is learned using data from the mixture distribution $D_{\bm{p}}$; Evolutionary fitness is associated with prediction accuracy; Differences in fitness drive change in mixture coefficients; Possible long-term tendencies are dominance (only $A$ survives), extinction (only $B$ survives), or coexistence (both survive).
  • Figure 2: Evolutionary dynamics for two groups, induced by an oracle linear classifier (\ref{['sec:retraining_optimal_classifiers']}). (Left) Data distributions in feature space. Dashed line demonstrates the optimal linear classifier $h_{\bm{p}}$ for the uniform mixture ${\bm{p}}=(0.5,0.5)$. (Center) Evolutionary prediction game $F_k({\bm{p}})=\mathrm{acc}_k(h_{\bm{p}})$. The game has two stable equilibria with single-group dominance, and one unstable coexistence equilibrium. Overall population accuracy $\mathrm{acc}_{\bm{p}}(h_{\bm{p}})$ is convex, and maximized at the boundries. (Right) Replicator dynamics induced by the game, for various initial states ${\bm{p}}^0$. Populations evolve towards fixed points with single-group dominance.
  • Figure 3: Graphical illustration of theoretical coexistence results. Top row presents data distributions, bottom row shows the corresponding prediction games. (Left) Stable and mutualistic coexistence induced by the use of a proxy loss (\ref{['thm:soft_svm_coexistence']}). (Center) Stable coexistence induced by interpolation (\ref{['subsec:knn_coexistence_proof']}). (Right) Mutualistic coexistence induced by a finite training set (\ref{['subsec:finite_data_coexistence_proof']}).
  • Figure 4: Empirical evaluation in two-group settings. (Left) Game induced by CIFAR-10 with Resnet-9 and groups representing horizontal flips. The game has unstable mutualism (\ref{['subsec:cifar_experiment']}). (Center) Simulated stabilization of replicator dynamics on the CIFAR-10 game (\ref{['subsec:stabilization']}). (Right) Game induced by MNIST with label noise, showing stable mutualism (\ref{['subsec:mnist_experiment']}).
  • Figure 5: Evolutionary dynamics in a three-group setting, induced by a linear SVM trained on the ACSIncome dataset (\ref{['subsec:fairness_empirical']}). (Left) Replicator dynamics on the three-group simplex. Red line indicates a trajectory starting from the uniform mixture, and background colors indicate basins of attraction. (Center) Population composition over time for the same trajectory. Proportion of NY users becomes negligible at $t \approx 316$. (Right) Fitness over time. Effective group disparity is significant at the outset, but shrinks by an order of magnitude after NY users are driven out.
  • ...and 9 more figures

Theorems & Definitions (78)

  • Definition 3.1: Evolutionary Prediction Game
  • Definition 3.2: Nash equilibrium of a population game; e.g. sandholm2010population
  • Proposition 3.1
  • Definition 4.1: Oracle classifier
  • Theorem 4.1
  • Theorem 5.1
  • Proposition 5.1
  • Proposition D.1: Expected loss is linear in ${\bm{p}}$ for a fixed hypothesis
  • proof
  • Definition D.1: Overall accuracy equality; verma2018fairness
  • ...and 68 more