Many-Objective Neuroevolution for Testing Games

TL;DR

This paper tackles automated test generation for games under high randomness by extending the neural-network-based test generator NEATEST to a many-objective optimisation framework. It integrates MOSA, MIO, and NEWS/D to target multiple program states in Scratch games, while preserving Neatest's neural encoding and robustness checks. Empirical results show that many-objective approaches improve average branch coverage from $75.88\%$ to $81.33\%$ and dramatically reduce search time by up to $93.28\%$, with MOSA-Neatest achieving the best overall performance. The findings demonstrate that casting game test generation as a many-objective neuroevolution problem yields faster, more comprehensive coverage and suggests promising directions for efficiency improvements and cross-domain applications.

Abstract

Generating tests for games is challenging due to the high degree of randomisation inherent to games and hard-to-reach program states that require sophisticated gameplay. The test generator NEATEST tackles these challenges by combining search-based software testing principles with neuroevolution to optimise neural networks that serve as test cases. However, since NEATEST is designed as a single-objective algorithm, it may require a long time to cover fairly simple program states or may even get stuck trying to reach unreachable program states. In order to resolve these shortcomings of NEATEST, this work aims to transform the algorithm into a many-objective search algorithm that targets several program states simultaneously. To this end, we combine the neuroevolution algorithm NEATEST with the two established search-based software testing algorithms, MIO and MOSA. Moreover, we adapt the existing many-objective neuroevolution algorithm NEWS/D to serve as a test generator. Our experiments on a dataset of 20 SCRATCH programs show that extending NEATEST to target several objectives simultaneously increases the average branch coverage from 75.88% to 81.33% while reducing the required search time by 93.28%.

Figures (3)

• Figure 1: The CreateYourWorld game with example code scripts.
• Figure 2: Achieved coverages over all projects when varying the secondary fitness criteria.
• Figure 3: Average program coverage achieved over time.