Competition and Diversity in Generative AI

TL;DR

The paper formalizes how competition among generative AI tools can counteract algorithmic monoculture by promoting output diversity, using a single- and multi-tool game with output types and negative externalities. It develops a rigorous model with ranking stability and a class of score functions $s_\gamma$, establishes existence and uniqueness of symmetric equilibria, and proves that stronger competition increases diversity while bounding welfare losses (PoA ≤ 2). The authors augment theory with Scattergories experiments using multiple LLMs and temperatures, showing competition shifts model rankings and enhances diversity in equilibrium, consistent with the theory. They also validate a key assumption about ranking stability across prompts and demonstrate that in competitive markets, even superior tools can be outcompeted, suggesting that tool developers should consider joint evaluation and niche differentiation. Overall, the work highlights the importance of evaluating generative AI across output distributions in competitive settings and discusses practical implications for alignment, data value, and platform design.

Abstract

Recent evidence, both in the lab and in the wild, suggests that the use of generative artificial intelligence reduces the diversity of content produced. The use of the same or similar AI models appears to lead to more homogeneous behavior. Our work begins with the observation that there is a force pushing in the opposite direction: compe- tition. When producers compete with one another (e.g., for customers or attention), they are incentivized to create novel or unique content. We explore the impact com- petition has on both content diversity and overall social welfare. Through a formal game-theoretic model, we show that competitive markets select for diverse AI models, mitigating monoculture. We further show that a generative AI model that performs well in isolation (i.e., according to a benchmark) may fail to provide value in a compet- itive market. Our results highlight the importance of evaluating generative AI models across the breadth of their output distributions, particularly when they will be deployed in competitive environments. We validate our results empirically by using language models to play Scattergories, a word game in which players are rewarded for answers that are both correct and unique. Overall, our results suggest that homogenization due to generative AI is unlikely to persist in competitive markets, and instead, competition in downstream markets may drive diversification in AI model development

Figures (16)

• Figure 1: Sample score functions. We plot $\frac{1}{s_\gamma(x)}$ to show a player's utility decreases as congestion increases. Larger values of $\gamma$ correspond to stronger penalties for collisions.
• Figure 2: For further intuition on \ref{['lem:n-to-infty']}, we show how $\mathbf{p}_{\textsc{opt}}(n, \mathbf{d}, s_\gamma)_1$ and $\mathbf{p}_{\textsc{eq}}(n, \mathbf{d}, s_\gamma)_1$ respectively vary with both $n$ and $\gamma$ for the instance given by $\mathbf{d} = [5; ~ 2]$. For $\gamma < 1$, both $\mathbf{p}_{\textsc{opt}}$ and $\mathbf{p}_{\textsc{eq}}$ converge to $\mathbf{p}^{\dagger}(\gamma)$, shown in magenta. For $\gamma \ge 1$, $\mathbf{p}_{\textsc{opt}}$ converges to the uniform distribution, with a discontinuity at $\gamma = 1$. Further, while $\mathbf{p}_{\textsc{eq}}$ is monotone in both $n$ and $\gamma$ (\ref{['thm:diversity-n', 'thm:diversity-gamma']}), $\mathbf{p}_{\textsc{opt}}$ is not. Increasing $n$ can lead to a less diverse symmetric socially optimal strategy.
• Figure 3: Per-player utility as a function of $\tau$ for different values of $n$. The highest line corresponds to $n=1$, and the lowest corresponds to $n=15$. $\gamma = 1.0$.
• Figure 4: Socially optimal and equilibrium temperatures for each language model as a function of $n$. Observe that (1) all curves are increasing in $n$, and (2) equilibrium temperatures are lower than their socially optimal counterparts. $\gamma = 1.0$
• Figure 5: Socially optimal and equilibrium welfare (per player) for each language model as a function of $n$. Observe that (1) welfare is decreasing in $n$, and (2) equilibrium welfare is not too far from optimal. phi3.5 and gemma2 perform better in the presence of stronger competition. $\gamma = 1.0$.

Theorems & Definitions (88)

• Definition 3.1: Score function
• Theorem 4.1
• Definition 4.2: Majorization
• Theorem 4.3: More players $\Longrightarrow$ more diverse equilibria
• Theorem 4.4: Greater externalities $\Longrightarrow$ more diverse equilibria
• Theorem 4.5: Optimal strategies are more diverse than equilibria
• Lemma 4.5: Strategies as $n \to \infty$
• Lemma 4.5: Equivalence when $\gamma = \infty$
• Lemma 4.5: $\gamma = \infty$ yields the most diverse equilibria
• Lemma 4.5: Asymmetric equilibria converge to symmetric equilibra