Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strategy Game-Playing with Size-Constrained State Abstraction

Linjie Xu, Diego Perez-Liebana, Alexander Dockhorn

TL;DR

Strategy games create very large search spaces that challenge MCTS-based planning. The authors propose size-constrained state abstraction (SCSA), which caps the number of ground states per abstract node to avoid mid-search abandonment and reduce hyperparameter sensitivity. Empirical tests in three Stratega games show SCSA outperforms baseline abstractions and achieves competitive results with Elastic MCTS in more complex settings, with open-sourced code at https://github.com/GAIGResearch/Stratega. The work analyzes compression rates to reveal a trade-off between memory usage and performance, and demonstrates robust, domain-general performance across variants.

Abstract

Playing strategy games is a challenging problem for artificial intelligence (AI). One of the major challenges is the large search space due to a diverse set of game components. In recent works, state abstraction has been applied to search-based game AI and has brought significant performance improvements. State abstraction techniques rely on reducing the search space, e.g., by aggregating similar states. However, the application of these abstractions is hindered because the quality of an abstraction is difficult to evaluate. Previous works hence abandon the abstraction in the middle of the search to not bias the search to a local optimum. This mechanism introduces a hyper-parameter to decide the time to abandon the current state abstraction. In this work, we propose a size-constrained state abstraction (SCSA), an approach that limits the maximum number of nodes being grouped together. We found that with SCSA, the abstraction is not required to be abandoned. Our empirical results on $3$ strategy games show that the SCSA agent outperforms the previous methods and yields robust performance over different games. Codes are open-sourced at https://github.com/GAIGResearch/Stratega.

Strategy Game-Playing with Size-Constrained State Abstraction

TL;DR

Strategy games create very large search spaces that challenge MCTS-based planning. The authors propose size-constrained state abstraction (SCSA), which caps the number of ground states per abstract node to avoid mid-search abandonment and reduce hyperparameter sensitivity. Empirical tests in three Stratega games show SCSA outperforms baseline abstractions and achieves competitive results with Elastic MCTS in more complex settings, with open-sourced code at https://github.com/GAIGResearch/Stratega. The work analyzes compression rates to reveal a trade-off between memory usage and performance, and demonstrates robust, domain-general performance across variants.

Abstract

Playing strategy games is a challenging problem for artificial intelligence (AI). One of the major challenges is the large search space due to a diverse set of game components. In recent works, state abstraction has been applied to search-based game AI and has brought significant performance improvements. State abstraction techniques rely on reducing the search space, e.g., by aggregating similar states. However, the application of these abstractions is hindered because the quality of an abstraction is difficult to evaluate. Previous works hence abandon the abstraction in the middle of the search to not bias the search to a local optimum. This mechanism introduces a hyper-parameter to decide the time to abandon the current state abstraction. In this work, we propose a size-constrained state abstraction (SCSA), an approach that limits the maximum number of nodes being grouped together. We found that with SCSA, the abstraction is not required to be abandoned. Our empirical results on strategy games show that the SCSA agent outperforms the previous methods and yields robust performance over different games. Codes are open-sourced at https://github.com/GAIGResearch/Stratega.
Paper Structure (16 sections, 1 equation, 6 figures, 4 tables, 2 algorithms)

This paper contains 16 sections, 1 equation, 6 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Stratega platform
  4. Background
  5. Monte Carlo Tree Search
  6. Monte Carlo Tree Search with Unit Ordering
  7. State Abstraction and Approximate MDP Homomorphism
  8. Elastic Monte Carlo Tree Search
  9. Method
  10. Experiments
  11. Agent Parameter Optimisation with NTBEA
  12. Performance on multi-unit-grid-based games
  13. Influence of abstract state size
  14. Compression Rate
  15. Conclusion and Future Work
  16. ...and 1 more sections

Figures (6)

  • Figure 1: A screenshot of Kill The King game. In this case, each player has one king, two warriors, two archers and two healers
  • Figure 2: A screenshot of Push Them All game. Each player has three pushers that try to push enemy units into holes.
  • Figure 3: A screenshot of Two Kingdom game. The blue side spawned a worker to mine the gold and the red side spawned three warriors to protect the king.
  • Figure 4: The number of opponents that the agent outperforms.
  • Figure 5: Performance of the SCSA agent with different values for its size constraint. Results of the SCSA agent playing against the corresponding Rule-based agents are visualized, including win rates and standard errors.
  • ...and 1 more figures