Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games

Patric Feldmeier, Gordon Fraser

TL;DR

The paper addresses the challenge of generating test inputs for games when standard objective-guided search yields deceptive or uninformative fitness landscapes. It proposes combining Neatest, a neuroevolution-based test generator, with novelty search to reward novel behaviours, quantified via a behavior archive and a cosine-based similarity distance. A Scratch-based case study on CreateYourWorld and PokéClicker demonstrates that novelty can increase coverage and help reach complex program states, with statistically significant improvements. The work contributes a practical integration of novelty search into game testing and points to future directions in multi-objective optimisation and fault-detection using diverse test suites.

Abstract

As games challenge traditional automated white-box test generators, the Neatest approach generates test suites consisting of neural networks that exercise the source code by playing the games. Neatest generates these neural networks using an evolutionary algorithm that is guided by an objective function targeting individual source code statements. This approach works well if the objective function provides sufficient guidance, but deceiving or complex fitness landscapes may inhibit the search. In this paper, we investigate whether the issue of challenging fitness landscapes can be addressed by promoting novel behaviours during the search. Our case study on two Scratch games demonstrates that rewarding novel behaviours is a promising approach for overcoming challenging fitness landscapes, thus enabling future research on how to adapt the search algorithms to best use this information.

Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games

TL;DR

The paper addresses the challenge of generating test inputs for games when standard objective-guided search yields deceptive or uninformative fitness landscapes. It proposes combining Neatest, a neuroevolution-based test generator, with novelty search to reward novel behaviours, quantified via a behavior archive and a cosine-based similarity distance. A Scratch-based case study on CreateYourWorld and PokéClicker demonstrates that novelty can increase coverage and help reach complex program states, with statistically significant improvements. The work contributes a practical integration of novelty search into game testing and points to future directions in multi-objective optimisation and fault-detection using diverse test suites.

Abstract

As games challenge traditional automated white-box test generators, the Neatest approach generates test suites consisting of neural networks that exercise the source code by playing the games. Neatest generates these neural networks using an evolutionary algorithm that is guided by an objective function targeting individual source code statements. This approach works well if the objective function provides sufficient guidance, but deceiving or complex fitness landscapes may inhibit the search. In this paper, we investigate whether the issue of challenging fitness landscapes can be addressed by promoting novel behaviours during the search. Our case study on two Scratch games demonstrates that rewarding novel behaviours is a promising approach for overcoming challenging fitness landscapes, thus enabling future research on how to adapt the search algorithms to best use this information.
Paper Structure (8 sections, 5 figures)

This paper contains 8 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Search-based Software Testing For Games via Neuroevolution
  3. Combining Test Generation for Games with the Search for Novelty
  4. Case Study
  5. Threats to Validity
  6. Results
  7. Related Work
  8. Conclusions

Figures (5)

  • Figure 1: The CreateYourWorld game together with the condition checking if the player has reached the next level by touching the orange portal on the right side of the level.
  • Figure 2: The PokéClicker game together with two conditions that are evaluated when pressing the Pokémon button and responsible for navigating the player to different menus.
  • Figure 3: Achieved coverage of Neatest and the proposed novelty approach over both dataset games.
  • Figure 4: Achieved coverage over time of Neatest and the proposed novelty approach for the PokéClicker game.
  • Figure 5: Achieved coverage over time of Neatest and the proposed novelty approach for the CreateYourWorld game.