Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs

Hidenori Tanaka

Abstract

Multi-agent systems powered by large language models (LLMs) are increasingly deployed in settings that shape consequential decisions, both directly and indirectly. Yet it remains unclear whether their outcomes reflect collective reasoning, systematic bias, or mere chance. Recent work has sharpened this question with naming games, showing that even when no individual agent favors any label a priori, populations rapidly break symmetry and reach consensus. Here, we reveal the mechanism by introducing a minimal model, Quantized Simplex Gossip (QSG), and trace the microscopic origin of this agreement to mutual in-context learning. In QSG, agents maintain internal belief states but learn from one another's sampled outputs, so one agent's arbitrary choice becomes the next agent's evidence and can compound toward agreement. By analogy with neutral evolution, we call this sampling-driven regime memetic drift. QSG predicts a crossover from a drift-dominated regime, where consensus is effectively a lottery, to a selection regime, where weak biases are amplified and shape the outcome. We derive scaling laws for drift-induced polarization as a function of population size, communication bandwidth, in-context adaptation rate, and agents' internal uncertainty, and we validate them in both QSG simulations and naming-game experiments with LLM populations. Together, these results provide a framework for studying the collective mechanisms of social representation formation in multi-agent systems.

When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs

Abstract

Multi-agent systems powered by large language models (LLMs) are increasingly deployed in settings that shape consequential decisions, both directly and indirectly. Yet it remains unclear whether their outcomes reflect collective reasoning, systematic bias, or mere chance. Recent work has sharpened this question with naming games, showing that even when no individual agent favors any label a priori, populations rapidly break symmetry and reach consensus. Here, we reveal the mechanism by introducing a minimal model, Quantized Simplex Gossip (QSG), and trace the microscopic origin of this agreement to mutual in-context learning. In QSG, agents maintain internal belief states but learn from one another's sampled outputs, so one agent's arbitrary choice becomes the next agent's evidence and can compound toward agreement. By analogy with neutral evolution, we call this sampling-driven regime memetic drift. QSG predicts a crossover from a drift-dominated regime, where consensus is effectively a lottery, to a selection regime, where weak biases are amplified and shape the outcome. We derive scaling laws for drift-induced polarization as a function of population size, communication bandwidth, in-context adaptation rate, and agents' internal uncertainty, and we validate them in both QSG simulations and naming-game experiments with LLM populations. Together, these results provide a framework for studying the collective mechanisms of social representation formation in multi-agent systems.

Paper Structure

This paper contains 35 sections, 2 theorems, 47 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Modeling: Quantized Simplex Gossip (QSG)
  3. QSG as a null model: explicit assumptions.
  4. Analysis and Scaling Laws
  5. Macroscopic observables
  6. Soft exchange preserves the mean in expectation and contracts disagreement
  7. Hard sampling injects an extra variance term
  8. Mean-field approximation and consensus time.
  9. Top-$m$ reduces the symmetry-breaking drift as $1/m$
  10. Absorption at $\alpha=1$ and run-level symmetry breaking for $\alpha<1$
  11. Weak asymmetry and the drift--selection crossover ($K=2$)
  12. Experimental Validation
  13. Discussion and Conclusion
  14. Additional Theory for Quantized Simplex Gossip (QSG)
  15. Proof of Theorem \ref{['thm:hard_variance']}: hard-sampling drift decomposition
  16. ...and 20 more sections

Key Result

Theorem 1

Consider QSG with adaptation rate $\alpha\in(0,1]$. Let $\bar{x}$ be the population mean and define the polarization potential $U\coloneqq \|\bar{x}\|_2^2$. Conditioned on the current state $X$, the expected one-step change in $U$ satisfies where the expectation is over the random choice of $(S,L)$ and (for Hard) the sample $k^\star$. The additional term in eq:extra_variance is the sampling varia

Figures (10)

  • Figure 1: Symmetry breaking in an LLM naming game (GPT-4o, $N=24$, $K=3$). (a) Mean coordination across trials (95% CI). (b) Representative label-frequency trajectories from three trials, one for each eventual winner. (c) Mean simplex trajectories conditioned on the eventual winning label.
  • Figure 2: Individual vs. mutual in-context learning. Top: standard in-context learning, where a single agent updates from i.i.d. tokens drawn from a stationary external distribution. Bottom: mutual in-context learning, where agents update from one another's sampled outputs, so the population becomes its own evolving data source.
  • Figure 3: Drift--selection crossover in multi-agent naming from GPT-4o experiments and QSG theory. Panels (a)--(c) show two-label (Dog, Cat) naming-game experiments with GPT-4o using random ordered speaker--listener pairs for a fixed referent $r$. (a) At $N=8$, runs show substantial run-to-run variability, and the inset shows winner counts across trials. (b) At $N=800$, a weak asymmetry consistently selects the same winner (Cat), again reflected in the inset counts. (c) Fixation probability versus $N$ shows a finite-size crossover. (d) The corresponding tempered-sampling crossover diagram derived from QSG theory in $(T,mN/\alpha)$; the black curve marks $\Gamma_T=1$, where $\Gamma_T=\frac{mN}{\alpha}\left|\frac{1}{T}-1\right|$.
  • Figure 4: Quantized Simplex Gossip (QSG): interaction protocol. 1. A population of $N$ agents, each with an internal state $x_i \in \Delta^{K-1}$. 2. An ordered speaker--listener pair is selected uniformly at random. 3. For a fixed referent, the speaker samples and sends a discrete message $y$ from $x_S$ in the quantized regimes (Hard or Top-$m$); Soft corresponds to transmitting the full distribution. 4. The listener updates toward the received message with adaptation rate $\alpha$, $x_L \leftarrow (1-\alpha)x_L + \alpha y$.
  • Figure 5: Probabilistic naming and sampling-noise geometry. (a) A referent is represented internally as a distribution over candidate labels. (b) Agent states lie on the simplex: near-uniform, high-entropy states generate larger sampling noise under quantization, whereas peaked, low-entropy states generate less. (c) Sampling-driven drift strength on the simplex, proportional to $1-\|x\|_2^2$, is maximal near the center and vanishes at the vertices.
  • ...and 5 more figures

Theorems & Definitions (2)

  • Theorem 1: Hard sampling increases polarization via sampling variance
  • Theorem 2: Top-$m$ drift term scales as $1/m$