SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models

TL;DR

SwarmBrain presents a two-tier LLM-based architecture for StarCraft II that separates macro strategic planning from micro tactical execution to overcome Real-Time Strategy latency. The Overmind Intelligence Matrix handles high-level decisions using LLM-led reasoning and a memory module, while the Swarm ReflexNet provides fast, autonomous state-machine behaviors for Zerg units. SC2 Brain translates Overmind concepts into structured in-game commands, and the Command Center ensures feasibility before execution. Empirical results show SwarmBrain defeating multiple Terran difficulties in most trials, with latency and execution gaps at higher difficulty, highlighting both the promise and current limits of LLM-driven RTS agents. The work emphasizes the importance of reducing reasoning delays and integrating visual information to enable robust, scalable performance in fast-paced multi-agent environments.

Abstract

Large language models (LLMs) have recently garnered significant accomplishments in various exploratory tasks, even surpassing the performance of traditional reinforcement learning-based methods that have historically dominated the agent-based field. The purpose of this paper is to investigate the efficacy of LLMs in executing real-time strategy war tasks within the StarCraft II gaming environment. In this paper, we introduce SwarmBrain, an embodied agent leveraging LLM for real-time strategy implementation in the StarCraft II game environment. The SwarmBrain comprises two key components: 1) a Overmind Intelligence Matrix, powered by state-of-the-art LLMs, is designed to orchestrate macro-level strategies from a high-level perspective. This matrix emulates the overarching consciousness of the Zerg intelligence brain, synthesizing strategic foresight with the aim of allocating resources, directing expansion, and coordinating multi-pronged assaults. 2) a Swarm ReflexNet, which is agile counterpart to the calculated deliberation of the Overmind Intelligence Matrix. Due to the inherent latency in LLM reasoning, the Swarm ReflexNet employs a condition-response state machine framework, enabling expedited tactical responses for fundamental Zerg unit maneuvers. In the experimental setup, SwarmBrain is in control of the Zerg race in confrontation with an Computer-controlled Terran adversary. Experimental results show the capacity of SwarmBrain to conduct economic augmentation, territorial expansion, and tactical formulation, and it shows the SwarmBrain is capable of achieving victory against Computer players set at different difficulty levels.

Figures (23)

• Figure 1: The framework of the interaction between SwarmBrain and StarCraft II environment.
• Figure 2: The framework of the Overmind Intelligence Matrix.
• Figure 3: The example map of "Automaton LE".
• Figure 4: The connection diagram between different mineral fields.
• Figure 5: A map matrix representation centered on mineral field locations.