Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Increasing intelligence in AI agents can worsen collective outcomes

Neil F. Johnson

TL;DR

It is shown empirically and mathematically that when resources are scarce, AI model diversity and reinforcement learning increase dangerous system overload, though tribe formation lessens this risk.

Abstract

When resources are scarce, will a population of AI agents coordinate in harmony, or descend into tribal chaos? Diverse decision-making AI from different developers is entering everyday devices -- from phones and medical devices to battlefield drones and cars -- and these AI agents typically compete for finite shared resources such as charging slots, relay bandwidth, and traffic priority. Yet their collective dynamics and hence risks to users and society are poorly understood. Here we study AI-agent populations as the first system of real agents in which four key variables governing collective behaviour can be independently toggled: nature (innate LLM diversity), nurture (individual reinforcement learning), culture (emergent tribe formation), and resource scarcity. We show empirically and mathematically that when resources are scarce, AI model diversity and reinforcement learning increase dangerous system overload, though tribe formation lessens this risk. Meanwhile, some individuals profit handsomely. When resources are abundant, the same ingredients drive overload to near zero, though tribe formation makes the overload slightly worse. The crossover is arithmetical: it is where opposing tribes that form spontaneously first fit inside the available capacity. More sophisticated AI-agent populations are not better: whether their sophistication helps or harms depends entirely on a single number -- the capacity-to-population ratio -- that is knowable before any AI-agent ships.

Increasing intelligence in AI agents can worsen collective outcomes

TL;DR

It is shown empirically and mathematically that when resources are scarce, AI model diversity and reinforcement learning increase dangerous system overload, though tribe formation lessens this risk.

Abstract

When resources are scarce, will a population of AI agents coordinate in harmony, or descend into tribal chaos? Diverse decision-making AI from different developers is entering everyday devices -- from phones and medical devices to battlefield drones and cars -- and these AI agents typically compete for finite shared resources such as charging slots, relay bandwidth, and traffic priority. Yet their collective dynamics and hence risks to users and society are poorly understood. Here we study AI-agent populations as the first system of real agents in which four key variables governing collective behaviour can be independently toggled: nature (innate LLM diversity), nurture (individual reinforcement learning), culture (emergent tribe formation), and resource scarcity. We show empirically and mathematically that when resources are scarce, AI model diversity and reinforcement learning increase dangerous system overload, though tribe formation lessens this risk. Meanwhile, some individuals profit handsomely. When resources are abundant, the same ingredients drive overload to near zero, though tribe formation makes the overload slightly worse. The crossover is arithmetical: it is where opposing tribes that form spontaneously first fit inside the available capacity. More sophisticated AI-agent populations are not better: whether their sophistication helps or harms depends entirely on a single number -- the capacity-to-population ratio -- that is knowable before any AI-agent ships.
Paper Structure (5 sections, 1 equation, 3 figures, 1 table)

This paper contains 5 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Agent architecture
  2. Decision pipeline
  3. Reward and adaptation
  4. Tribal dynamics (L5/LOTF only)
  5. Experimental parameters

Figures (3)

  • Figure 1: Technology ladder: system overload across five levels of population sophistication. Under abundance ($C/N > 0.6$), the most sophisticated L4/L5 achieve near-zero overload. Under scarcity ($C/N \lesssim 0.5$), the cheapest population (L1) achieves the lowest overload. All five curves cross near $C/N \approx 0.5$. L1 and L2 can be calculated analytically; L3--L5 are empirical (20 seeds $\times$ 500 rounds, $\pm 1$ SE shaded bands). Inset: L4 (FRD)--L5 (LOTF) crossover at $N = 15$ ($\Delta = -11.9\pm0.9$ pp at $C/N = 0.40$; $\Delta = +8.9\pm1.1$ pp at $C/N = 0.67$; SI Appendix I).
  • Figure 2: Individual AI-agent win rates. Win rates for followers in L5 (LOTF) and L4 (FRD) trace a strong U-shape, while anti-followers in both systems trace a weak inverted U-shape. Four curves: two dispositions $\times$ two experiments. Empirical: 20 seeds $\times$ 500 rounds; $\pm 1$ SE shaded bands.
  • Figure 3: Tribal membership dynamics at $C = 2$ (representative seed). Each row is one AI-agent (labelled by model and initial disposition); colour indicates tribe membership over 500 rounds. All agents start in a single tribe (grey). Within ${\sim}50$ rounds, dispositional polarisation drives fission into opposing factions: the three high-$p$ agents (GPT-2 family) and three low-$p$ agents (Pythia + OPT-125M) sort into distinct tribes, with OPT-350M ($p_0 = 0.50$) as a persistent singleton. OPT-125M (initial $p_0 = 0.33$, Meta's OPT family) consistently joins the Pythia anti-follower bloc rather than its architectural sibling OPT-350M --- cross-family sorting driven by disposition, not lineage. Tribe identity labels change through fission--fusion events, but the $3{+}3{+}1$ partition structure is quite stable. Dashed vertical line: conch break at round 250 (dynamically irrelevant; SI Appendix F). Full tribal dynamics for $C = 1$--$6$ are shown in SI Appendix K.