Memory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Games
Runnan Qi, Yanan Ni, Lumin Jiang, Zongyuan Li, Kuihua Huang, Xian Guo
TL;DR
The paper tackles hallucinations and fragmented decision-making in LLM-based real-time strategy agents. It introduces Memory-Augmented State Machine Prompting (MASMP), which combines state-machine prompting with a strategic memory module, formalized by $ (s_t, a_t) ~ ext{LLM ext_Generate}(o_t, M_{t-1}, prompt_{sm}) $ and $ M_t = ext{Update}(M_{t-1}, s_t) $. In StarCraft II experiments, MASMP achieves a 60% win rate against the hardest built-in AI (Lv7), vastly outperforming baselines, and it demonstrates improved interpretability and reliability through FSM-like decision-making. The approach advances neuro-symbolic AI for complex decision-making tasks and suggests potential for multi-agent coordination and cross-domain applications.
Abstract
This paper proposes Memory-Augmented State Machine Prompting (MASMP), a novel framework for LLM agents in real-time strategy games. Addressing key challenges like hallucinations and fragmented decision-making in existing approaches, MASMP integrates state machine prompting with memory mechanisms to unify structured actions with long-term tactical coherence. The framework features: (1) a natural language-driven state machine architecture that guides LLMs to emulate finite state machines and behavior trees through prompts, and (2) a lightweight memory module preserving strategic variables (e.g., tactics, priority units) across decision cycles. Experiments in StarCraft II demonstrate MASMP's 60% win rate against the hardest built-in AI (Lv7), vastly outperforming baselines (0%). Case studies reveal the method retains LLMs' semantic comprehension while resolving the "Knowing-Doing Gap" through strict state-action mapping, achieving both interpretability and FSM-like reliability. This work establishes a new paradigm for combining neural and symbolic AI in complex decision-making.