Gamifying a Software Testing Course with Continuous Integration

TL;DR

The paper addresses low motivation for software testing education by embedding gamification into the continuous integration (CI) workflow using a Jenkins plugin called Gamekins. Gamekins generates and manages challenges, quests, leaderboards, and achievements from code coverage and mutation analysis to incentivize test writing and improvement. In a undergraduate software testing course, the authors observe correlations between Gamekins activity and testing behaviors and report a significant improvement in correctness relative to a non-gamified iteration, along with positive student feedback. The work demonstrates that CI-integrated gamification can meaningfully enhance testing practices and engagement, and it outlines concrete avenues for refinement and broader deployment.

Abstract

Testing plays a crucial role in software development, and it is essential for software engineering students to receive proper testing education. However, motivating students to write tests and use automated testing during software development can be challenging. To address this issue and enhance student engagement in testing when they write code, we propose to incentivize students to test more by gamifying continuous integration. For this we use Gamekins, a tool that is seamlessly integrated into the Jenkins continuous integration platform and uses game elements based on commits to the source code repository: Developers can earn points by completing test challenges and quests generated by Gamekins, compete with other developers or teams on a leaderboard, and receive achievements for their test-related accomplishments. In this paper, we present our integration of Gamekins into an undergraduate-level course on software testing. We observe a correlation between how students test their code and their use of Gamekins, as well as a significant improvement in the accuracy of their results compared to a previous iteration of the course without gamification. As a further indicator of how this approach improves testing behavior, the students reported enjoyment in writing tests with Gamekins.

Figures (12)

• Figure 1: This overview highlights the current challenges, including a Line Coverage, a Method Coverage, and a Mutation Challenge. The Mutation Challenge is further explained with a code snippet, the mutated line of code, and a description.
• Figure 2: Example quest consisting of three Smell Challenges, with the active one highlighted
• Figure 3: The project leaderboard displays the completed challenges, as well as (unfinished) quests and achievements. It also shows the score and avatar of each user. Clicking on their name will take you to their achievements..
• Figure 4: The list of achievements in Gamekins is categorized into completed, unsolved, and secret ones. Each achievement has a title, a description, a date when it was solved, and an icon. Once solved, its icon is colored to indicate completion.
• Figure 5: Statistics on the use of Gamekins in both projects