Online automatic code generation for robot swarms: LLMs and self-organizing hierarchy

TL;DR

The paper tackles the problem of online, adaptive code generation for robot swarms by leveraging a self-organizing nervous system (SoNS) to provide global swarm estimation and temporarily centralized control. It proposes SoNS as middleware that enables dynamic hierarchies, allowing large-language models (LLMs) to generate and remotely update swarm behavior in real time. The authors demonstrate this approach in mixed aerial-ground swarms, achieving an 85% success rate across real and simulated trials and showing that on-the-fly LLM-generated Lua code can guide the swarm past obstacles. The work highlights the potential for scalable, adaptive swarm programming with SoNS-augmented architectures and outlines avenues for improving safety, code separation, and task complexity.

Abstract

Our recently introduced self-organizing nervous system (SoNS) provides robot swarms with 1) ease of behavior design and 2) global estimation of the swarm configuration and its collective environment, facilitating the implementation of online automatic code generation for robot swarms. In a demonstration with 6 real robots and simulation trials with >30 robots, we show that when a SoNS-enhanced robot swarm gets stuck, it can automatically solicit and run code generated by an external LLM on the fly, completing its mission with an 85% success rate.

Figures (2)

• Figure 1: Key frames from videos of the demo with real robots and an example successful simulation trial. Robots begin with no code for obstacle avoidance; they simply move forward while remaining in a square formation shape. When the ground robots become physically obstructed, one robot sends its available information to an external LLM, with a generic request for new code for all robots in the swarm. Once the new code has been received and all robots in the swarm have updated their programs in a self-organized manner, the ground robots successfully circumvent the obstacles.
• Figure 2: Task duration using online LLM-based code generation, 20 trials. In successful trials, the elapsed time spans from when the first robot reaches the first obstacle, to when the last robot surpasses the last obstacle. Constituent steps include: 1) robots try and fail repeatedly to move forward; 2) robots send request for help to LLM; 3) robots receive and execute generated code; 4) robots get unstuck and surpass the obstacles, thus completing the task. Out of 20 trials, robots failed to complete the task in three (red bar).