Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Emergence of Internal State-Modulated Swarming in Multi-Agent Patch Foraging System

Siddharth Chaturvedi, Ahmed EL-Gazzar, Marcel van Gerven

TL;DR

The paper addresses how internal-state modulation can induce swarming in a multi-agent patch-foraging setting with partial observability. It proposes a CTRNN-based velocity controller trained with CMA-ES, employing concurrent evaluation across agents to capture interaction effects. The results show adaptive foraging and swarming that persists even without patches, with aggregation strength modulated by a forager's internal resource level, supported by ablations and CTRNN-state analyses that indicate an urgency-like mechanism. This work links swarming to risk-sensitive foraging principles and demonstrates a sensorimotor coupling where internal state signals influence collective dynamics, with implications for decentralized control in noisy environments.

Abstract

Active particles are entities that sustain persistent out-of-equilibrium motion by consuming energy. Under certain conditions, they exhibit the tendency to self-organize through coordinated movements, such as swarming via aggregation. While performing non-cooperative foraging tasks, the emergence of such swarming behavior in foragers, exemplifying active particles, has been attributed to the partial observability of the environment, in which the presence of another forager can serve as a proxy signal to indicate the potential presence of a food source or a resource patch. In this paper, we validate this phenomenon by simulating multiple self-propelled foragers as they forage from multiple resource patches in a non-cooperative manner. These foragers operate in a continuous two-dimensional space with stochastic position updates and partial observability. We evolve a shared policy in the form of a continuous-time recurrent neural network that serves as a velocity controller for the foragers. To this end, we use an evolutionary strategy algorithm wherein the different samples of the policy-distribution are evaluated in the same rollout. Then we show that agents are able to learn to adaptively forage in the environment. Next, we show the emergence of swarming in the form of aggregation among the foragers when resource patches are absent. We observe that the strength of this swarming behavior appears to be inversely proportional to the amount of resource stored in the foragers, which supports the risk-sensitive foraging claims. Empirical analysis of the learned controller's hidden states in minimal test runs uncovers their sensitivity to the amount of resource stored in a forager. Clamping these hidden states to represent a lesser amount of resource hastens its learned aggregation behavior.

Emergence of Internal State-Modulated Swarming in Multi-Agent Patch Foraging System

TL;DR

The paper addresses how internal-state modulation can induce swarming in a multi-agent patch-foraging setting with partial observability. It proposes a CTRNN-based velocity controller trained with CMA-ES, employing concurrent evaluation across agents to capture interaction effects. The results show adaptive foraging and swarming that persists even without patches, with aggregation strength modulated by a forager's internal resource level, supported by ablations and CTRNN-state analyses that indicate an urgency-like mechanism. This work links swarming to risk-sensitive foraging principles and demonstrates a sensorimotor coupling where internal state signals influence collective dynamics, with implications for decentralized control in noisy environments.

Abstract

Active particles are entities that sustain persistent out-of-equilibrium motion by consuming energy. Under certain conditions, they exhibit the tendency to self-organize through coordinated movements, such as swarming via aggregation. While performing non-cooperative foraging tasks, the emergence of such swarming behavior in foragers, exemplifying active particles, has been attributed to the partial observability of the environment, in which the presence of another forager can serve as a proxy signal to indicate the potential presence of a food source or a resource patch. In this paper, we validate this phenomenon by simulating multiple self-propelled foragers as they forage from multiple resource patches in a non-cooperative manner. These foragers operate in a continuous two-dimensional space with stochastic position updates and partial observability. We evolve a shared policy in the form of a continuous-time recurrent neural network that serves as a velocity controller for the foragers. To this end, we use an evolutionary strategy algorithm wherein the different samples of the policy-distribution are evaluated in the same rollout. Then we show that agents are able to learn to adaptively forage in the environment. Next, we show the emergence of swarming in the form of aggregation among the foragers when resource patches are absent. We observe that the strength of this swarming behavior appears to be inversely proportional to the amount of resource stored in the foragers, which supports the risk-sensitive foraging claims. Empirical analysis of the learned controller's hidden states in minimal test runs uncovers their sensitivity to the amount of resource stored in a forager. Clamping these hidden states to represent a lesser amount of resource hastens its learned aggregation behavior.
Paper Structure (13 sections, 11 equations, 9 figures, 1 table, 1 algorithm)

This paper contains 13 sections, 11 equations, 9 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Model
  3. Position model
  4. Resource model
  5. Controller model
  6. Foraging objective
  7. Learning algorithm
  8. Simulation details
  9. Experiments and results
  10. Emergence of adaptive foraging
  11. Emergence of swarming
  12. CTRNN analysis
  13. Discussion and conclusion

Figures (9)

  • Figure 1: Environment snapshot, and local sensing limitations in the patch-foraging task. (b) Snapshot from one rollout with $n=300$ foragers and $m=400$ resource patches. Arrows indicate foragers' directions of gaze and movement. (a) Close up: forager $i=1$ emits equiangular rays; because its center lies within forager $i=2$, that forager is not registered; patch $j=1$ is detected by a ray, while forager $i=3$ falls in a blind spot between rays. (c) Close up: patch $j=2$ is occluded by forager $i=5$ relative to forager $i=4$; patch $j=3$ lies beyond the ray range.
  • Figure 2: (a) The progression of mean, maximum and minimum fitness across generations during evolution. (b) Trajectories of $n=10$ foragers in a small test environment containing $m=10$ resource patches before optimization begins simulated for $T=2{,}000$ steps. (c) Trajectories of $n=10$ foragers in a small test environment containing $m=10$ resource patches with same initial conditions as (b) after optimization simulated for $T=2{,}000$ steps.
  • Figure 3: The number of unique patches visited by different foragers while following trajectories of Figure \ref{['fig:fitness']}(c).
  • Figure 4: Swarming emerges without patches and disappears when inter-agent sensing is ablated. (a) Ten foragers in a small test environment self-aggregate under the learned policy. Trajectories simulated for $T=2{,}000$ steps. (b) With inter-agent sensing removed, under identical initial conditions as (a), agents remain dispersed and execute small local orbits near their spawn points. (c) Mean nearest neighbor distance (MNN) over time: under non-ablated (intact) conditions, it declines and stabilizes ($\approx 17$ units), whereas under ablated conditions, it stays high ($\approx 36$ units).
  • Figure 5: Progression of mean nearest neighbor (MNN) distance (calculated for $n=10$ foragers) plotted against time steps for different clamped resource levels $\bar{e}$. The plots are smoothed with a 125-step moving average filter.
  • ...and 4 more figures