The Hive Mind is a Single Reinforcement Learning Agent

TL;DR

The paper shows that imitation-based collective decision-making in swarms can be mathematically mapped to reinforcement learning in a single macro-agent, where a hive mind learns via Maynard-Cross Learning (MCL) and its variants. By connecting Cross Learning (CL) and the Maynard-Smith/MRD frameworks, it unifies population dynamics (Taylor and Maynard-Smith replicator dynamics) with online RL in both streaming and parallel settings. Through theoretical derivations and extensive simulations, it demonstrates that a swarm of simple, non-learning individuals can implement a scalable, parallel reinforcement-learning process, enabling fast adaptation and robust collective intelligence. The findings have broad implications for biology, economics, swarm robotics, and algorithmic design, suggesting that group-level learning emerges from simple imitation rules and can be leveraged to design scalable, adaptive collective systems.

Abstract

Decision-making is an essential attribute of any intelligent agent or group. Natural systems are known to converge to optimal strategies through at least two distinct mechanisms: collective decision-making via imitation of others, and individual trial-and-error. This paper establishes an equivalence between these two paradigms by drawing from the well-established collective decision-making model of nest-hunting in swarms of honey bees. We show that the emergent distributed cognition (sometimes referred to as the $\textit{hive mind}$) arising from individual bees following simple, local imitation-based rules is that of a single online reinforcement learning (RL) agent interacting with many parallel environments. The update rule through which this macro-agent learns is a bandit algorithm that we coin $\textit{Maynard-Cross Learning}$. Our analysis implies that a group of cognition-limited organisms can be equivalent to a more complex, reinforcement-enabled entity, substantiating the idea that group-level intelligence may explain how seemingly simple and blind individual behaviors are selected in nature. From a biological perspective, this analysis suggests how such imitation strategies evolved: they constitute a scalable form of reinforcement learning at the group level, aligning with theories of kin and group selection. Beyond biology, the framework offers new tools for analyzing economic and social systems where individuals imitate successful strategies, effectively participating in a collective learning process. In swarm intelligence, our findings will inform the design of scalable collective systems in artificial domains, enabling RL-inspired mechanisms for coordination and adaptability at scale.

Figures (7)

• Figure 1: The "hive mind" of a swarm of $N$ bees nest-hunting among $n$ options is a single $n$-armed bandit RL agent learning from $N$ environments in parallel.
• Figure 2: Simulations: (a) Swarms of bees reach consensus more rapidly when following $R_{\text{wvoter}}$ collectively than when individual bees learn via $\alpha$-MCL. (b,c) Varying swarm and neighborhood sizes show that the theoretical predictions hold under practical constraints. (d) Simple variants of $R_{\text{success}}$ and $R_{\text{wvoter}}$ can surpass $R_{\text{wvoter}}$ in this environment, raising the question of why evolution favored $R_{\text{wvoter}}$ over alternative decision-making strategies.
• Figure 3: Results for streaming RL experiments.
• Figure 4: Results for parallel RL experiments.
• Figure 5: Results for population experiments.

Theorems & Definitions (7)

• definition 1
• remark 1
• definition 2
• definition 3
• proof
• definition 4
• proof