Engaging Developers in Exploratory Unit Testing through Gamification
Philipp Straubinger, Gordon Fraser
TL;DR
The paper investigates embedding gamification of exploratory unit testing directly into the IDE to boost developer engagement and improve test quality. It introduces the Gamekins IntelliJ plugin, which presents challenges, quests, achievements, and leaderboards to guide and reward exploratory testing within regular coding tasks. In a 60-minute controlled study using a simplified Defects4J-enabled Apache Commons CLI project, participants produced test suites with high line coverage ($\sim 90\%$) and strong mutation scores, detecting 11 of 14 real-world bugs, while reporting positive experiences overall. The results suggest that IDE-based gamification can increase participation and software quality, with future work focusing on richer challenges and a calmer notification design to support broader adoption in real-world settings.
Abstract
Exploratory testing, known for its flexibility and ability to uncover unexpected issues, often faces challenges in maintaining systematic coverage and producing reproducible results. To address these challenges, we investigate whether gamification of testing directly in the Integrated Development Environment (IDE) can guide exploratory testing. We therefore show challenges and quests generated by the Gamekins gamification system to make testing more engaging and seamlessly blend it with regular coding tasks. In a 60-minute experiment, we evaluated Gamekins' impact on test suite quality and bug detection. The results show that participants actively interacted with the tool, achieving nearly 90% line coverage and detecting 11 out of 14 bugs. Additionally, participants reported enjoying the experience, indicating that gamification can enhance developer participation in testing and improve software quality.