Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Curiosity Driven Multi-agent Reinforcement Learning for 3D Game Testing

Raihana Ferdous, Fitsum Kifetew, Davide Prandi, Angelo Susi

TL;DR

This paper addresses automated testing of 3D games by leveraging curiosity-driven cooperative multi-agent reinforcement learning. It introduces cMarlTest, which deploys active and passive agents to maximize diverse game-area coverage using a novelty-based reward and memory-aware state updates. Across Lab Recruits levels of varying size, cMarlTest generally achieves higher entity, entity-connection, and spatial coverage more efficiently than a single-agent baseline, though large levels remain challenging and benefit from increased budgets. The work demonstrates the practicality of MARL for game testing and outlines avenues for extending agent roles, incorporating deeper RL methods, and exploring bug-detection opportunities in complex 3D environments.

Abstract

Recently testing of games via autonomous agents has shown great promise in tackling challenges faced by the game industry, which mainly relied on either manual testing or record/replay. In particular Reinforcement Learning (RL) solutions have shown potential by learning directly from playing the game without the need for human intervention. In this paper, we present cMarlTest, an approach for testing 3D games through curiosity driven Multi-Agent Reinforcement Learning (MARL). cMarlTest deploys multiple agents that work collaboratively to achieve the testing objective. The use of multiple agents helps resolve issues faced by a single agent approach. We carried out experiments on different levels of a 3D game comparing the performance of cMarlTest with a single agent RL variant. Results are promising where, considering three different types of coverage criteria, cMarlTest achieved higher coverage. cMarlTest was also more efficient in terms of the time taken, with respect to the single agent based variant.

Curiosity Driven Multi-agent Reinforcement Learning for 3D Game Testing

TL;DR

This paper addresses automated testing of 3D games by leveraging curiosity-driven cooperative multi-agent reinforcement learning. It introduces cMarlTest, which deploys active and passive agents to maximize diverse game-area coverage using a novelty-based reward and memory-aware state updates. Across Lab Recruits levels of varying size, cMarlTest generally achieves higher entity, entity-connection, and spatial coverage more efficiently than a single-agent baseline, though large levels remain challenging and benefit from increased budgets. The work demonstrates the practicality of MARL for game testing and outlines avenues for extending agent roles, incorporating deeper RL methods, and exploring bug-detection opportunities in complex 3D environments.

Abstract

Recently testing of games via autonomous agents has shown great promise in tackling challenges faced by the game industry, which mainly relied on either manual testing or record/replay. In particular Reinforcement Learning (RL) solutions have shown potential by learning directly from playing the game without the need for human intervention. In this paper, we present cMarlTest, an approach for testing 3D games through curiosity driven Multi-Agent Reinforcement Learning (MARL). cMarlTest deploys multiple agents that work collaboratively to achieve the testing objective. The use of multiple agents helps resolve issues faced by a single agent approach. We carried out experiments on different levels of a 3D game comparing the performance of cMarlTest with a single agent RL variant. Results are promising where, considering three different types of coverage criteria, cMarlTest achieved higher coverage. cMarlTest was also more efficient in terms of the time taken, with respect to the single agent based variant.

Paper Structure

This paper contains 19 sections, 1 equation, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Running Example
  3. cMarlTest: Curiosity Driven MARL for Game Testing
  4. States and actions
  5. Agent roles
  6. Reward function
  7. Methodology
  8. Evaluation Setup
  9. Prototype
  10. Game levels
  11. Metrics
  12. Experiment protocol
  13. Results and Discussion
  14. Small size levels
  15. Medium size levels
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Level L1 in Lab Recruits.
  • Figure 2: Entity coverage, entity connection coverage and time boxplots
  • Figure 3: Spatial coverage for level S1 of Small size. The darker the color the less explored the area.
  • Figure 4: Spatial coverage for level M3 of Medium size. The darker the color the less explored the area.
  • Figure 5: Spatial coverage for level L3 of Large size. The darker the color the less explored the area.