Hierarchical Expert Prompt for Large-Language-Model: An Approach Defeat Elite AI in TextStarCraft II for the First Time

TL;DR

The paper tackles the challenge of using large language models for high-stakes, real-time decision-making in a complex environment (TextStarCraft II) where traditional RL approaches require extensive data and computation. It proposes the Hierarchical Expert Prompt (HEP), combining an Expert Tactic Prompt (ETP) with a Hierarchical Decision Prompt (HDP) to inject expert tactics and enforce priority-aware decision-making, enabling the LLM to handle tasks of varying importance. Empirical results show that HEP significantly improves decision quality, boosts economy, tech, and military capabilities, and, crucially, defeats Elite AI for the first time in this setting, with ablations confirming the necessity of both modules. The work demonstrates the practical potential of LLM-based decision-making in complex, hierarchically structured tasks and provides open-source resources to extend this approach to other domains.

Abstract

Since the emergence of the Large Language Model (LLM), LLM has been widely used in fields such as writing, translating, and searching. However, there is still great potential for LLM-based methods in handling complex tasks such as decision-making in the StarCraft II environment. To address problems such as lack of relevant knowledge and poor control over subtasks of varying importance, we propose a Hierarchical Expert Prompt (HEP) for LLM. Our method improves the understanding of game situations through expert-level tactical knowledge, improving the processing quality of tasks of varying importance through a hierarchical framework. Our approach defeated the highest level (Elite) standard built-in agent in TextStarCraft II for the first time and consistently outperformed the baseline method in other difficulties. Our experiments suggest that the proposed method is a practical solution for tackling complex decision-making challenges. The replay video can be viewed on https://www.bilibili.com/video/BV1uz42187EF and https://youtu.be/dO3PshWLV5M, and our codes have been open-sourced on https://github.com/luchang1113/HEP-LLM-play-StarCraftII.

Figures (39)

• Figure 1: StarCraft II. In this decision-making environment, players need to control units, collect resources, build and upgrade technology, and confront opponents with incomplete observation information, making it one of the most complex decision-making environments.
• Figure 2: Interacting with LLM: Hierarchical Expert Prompt Method in TextStarCraft II. LLM takes the L1 summaryBaseline (a highly condensed text observation) as input, obtains knowledge from Hierarchical Expert Prompt, and generates text actions according to analyses.
• Figure 3: Screenshots of Game Replay Against VeryHard Opponent.
• Figure 4: Resource Data. (a)(b) In the first 8 minutes, our method focused on collecting minerals and the amount of gas was low, after 8 minutes, our method started to collect a lot of gas to provide resources for building the army in the later stages. (c)(d) Our method is able to train more Probes compared to the baseline.
• Figure 5: Supply Data. (a)(b) Our method is able to use more supply compared to baseline, importantly, the army supply has a larger increase in the early game, makes it possible to defend the opponent's early attack and scouting. (c)(d) Our method continuously built Pylons to prevent running out of supply. During 400s to 600s, the baseline method struggled in increasing supply, while our method kept the supply increasing continuous.