Lifelong Evolution of Swarms

TL;DR

This work integrates lifelong learning with swarm evolution by introducing a lifelong evolutionary framework where a population of swarm controllers evolves under dynamically changing tasks. The key idea is that population diversity can retain knowledge of previous tasks and enable faster adaptation when tasks reappear, while the top individual may forget prior tasks unless regularization is applied. To address forgetting, the authors introduce a genetic distance regularization term, inspired by Elastic Weight Consolidation, that ties the current population to a reference model from the previous task. Empirical results on a NEAT-based swarm foraging task show rapid adaptation and partial retention at the population level, with model-specific regularization significantly reducing forgetting for the top individual, albeit with some task-performance trade-offs. The findings illuminate the potential of combining evolutionary search with lifelong learning principles to build robust, adaptive swarm systems in dynamic environments and open avenues for replay-based or archive-informed lifelong strategies.

Abstract

Adapting to task changes without forgetting previous knowledge is a key skill for intelligent systems, and a crucial aspect of lifelong learning. Swarm controllers, however, are typically designed for specific tasks, lacking the ability to retain knowledge across changing tasks. Lifelong learning, on the other hand, focuses on individual agents with limited insights into the emergent abilities of a collective like a swarm. To address this gap, we introduce a lifelong evolutionary framework for swarms, where a population of swarm controllers is evolved in a dynamic environment that incrementally presents novel tasks. This requires evolution to find controllers that quickly adapt to new tasks while retaining knowledge of previous ones, as they may reappear in the future. We discover that the population inherently preserves information about previous tasks, and it can reuse it to foster adaptation and mitigate forgetting. In contrast, the top-performing individual for a given task catastrophically forgets previous tasks. To mitigate this phenomenon, we design a regularization process for the evolutionary algorithm, reducing forgetting in top-performing individuals. Evolving swarms in a lifelong fashion raises fundamental questions on the current state of deep lifelong learning and on the robustness of swarm controllers in dynamic environments.

Figures (13)

• Figure 1: Overview of the lifelong evolutionary swarms framework. The swarm, composed of multiple agents, interacts with the environment through actions based on local sensor inputs (environment state) processed by an identical internal controller. Each agent keeps a copy of the controller. The sequence of actions of an agent determines the fitness of the controller. The evolutionary algorithm updates the controller based on their fitness. Periodically, the dynamic environment changes the underlying task which requires the swarm to adapt to novel conditions without forgetting the previous knowledge.
• Figure 2: Fitness evolution of the best individual in the population for each generation and across three tasks: red task (generations 0–200), blue task (generations 200–400), and a return to the red task (generations 400–600). Line colors correspond to the task, and vertical dashed lines mark the transitions between tasks. The rapid recovery of fitness after task switches highlights the system's adaptability and transfer of knowledge between related tasks.
• Figure 3: Current performance (solid lines) and retention at the population level (dashed lines) across task drift (line colors correspond to task colors). Population naturally preserves past knowledge while adapting to new objectives.
• Figure 4: Current performance (solid lines) and retention at the individual level (dashed lines) across task drift (line colors correspond to task colors). Retention fitness demonstrates catastrophic forgetting, as the best-performing individual on the current task fails to retain knowledge of prior tasks.
• Figure 5: Current performance (solid lines) and retention at the individual level (dashed lines) for the red and green tasks when applying a fixed regularization coefficient, $\lambda = 11$. Retention is higher than without regularization (mitigating forgetting), though at the cost of a reduced performance on the green task.