Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges

Xinrun Xu, Yuxin Wang, Chaoyi Xu, Ziluo Ding, Jiechuan Jiang, Zhiming Ding, Börje F. Karlsson

TL;DR

The paper surveys the use of language-based agents in complex digital games, outlining a perception-inference-action framework and a taxonomy of architectures. It synthesizes representative works across perception, memory, learning, reasoning, decision-making, and action, and highlights recurring challenges such as hallucination, error correction, generalization, and interpretability. By detailing methodological approaches and future directions, the work offers a roadmap for advancing LMAs in gaming and related AGI research. The insights aim to accelerate development of realistic, adaptable, and tool-augmented game agents with broad practical impact.

Abstract

The swift evolution of Large-scale Models (LMs), either language-focused or multi-modal, has garnered extensive attention in both academy and industry. But despite the surge in interest in this rapidly evolving area, there are scarce systematic reviews on their capabilities and potential in distinct impactful scenarios. This paper endeavours to help bridge this gap, offering a thorough examination of the current landscape of LM usage in regards to complex game playing scenarios and the challenges still open. Here, we seek to systematically review the existing architectures of LM-based Agents (LMAs) for games and summarize their commonalities, challenges, and any other insights. Furthermore, we present our perspective on promising future research avenues for the advancement of LMs in games. We hope to assist researchers in gaining a clear understanding of the field and to generate more interest in this highly impactful research direction. A corresponding resource, continuously updated, can be found in our GitHub repository.

A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges

TL;DR

The paper surveys the use of language-based agents in complex digital games, outlining a perception-inference-action framework and a taxonomy of architectures. It synthesizes representative works across perception, memory, learning, reasoning, decision-making, and action, and highlights recurring challenges such as hallucination, error correction, generalization, and interpretability. By detailing methodological approaches and future directions, the work offers a roadmap for advancing LMAs in gaming and related AGI research. The insights aim to accelerate development of realistic, adaptable, and tool-augmented game agents with broad practical impact.

Abstract

The swift evolution of Large-scale Models (LMs), either language-focused or multi-modal, has garnered extensive attention in both academy and industry. But despite the surge in interest in this rapidly evolving area, there are scarce systematic reviews on their capabilities and potential in distinct impactful scenarios. This paper endeavours to help bridge this gap, offering a thorough examination of the current landscape of LM usage in regards to complex game playing scenarios and the challenges still open. Here, we seek to systematically review the existing architectures of LM-based Agents (LMAs) for games and summarize their commonalities, challenges, and any other insights. Furthermore, we present our perspective on promising future research avenues for the advancement of LMs in games. We hope to assist researchers in gaining a clear understanding of the field and to generate more interest in this highly impactful research direction. A corresponding resource, continuously updated, can be found in our GitHub repository.
Paper Structure (25 sections, 3 figures)

This paper contains 25 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Perception
  3. Semantics
  4. Vision
  5. Inference
  6. Memory
  7. Common Sense.
  8. Game Background Knowledge.
  9. Retrieval.
  10. Learning
  11. Reasoning
  12. Decision-making
  13. Multi-Hop Inference.
  14. Long-Term Planning.
  15. Reflection.
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Target areas and Challenges leveraging Large Model capabilities in Game Playing Agents.
  • Figure 2: Taxonomy of research on complex digital game playing agents and large models.
  • Figure 3: Overview of Representative Works on Game-Playing Agents and Large Models.