Understanding Gaming the System by Analyzing Self-Regulated Learning in Think-Aloud Protocols
Jiayi Zhang, Conrad Borchers, Canwen Wang, Vishal Kumar, Leah Teffera, Bruce M. McLaren, Ryan S. Baker
TL;DR
This study investigates gaming the system in digital learning by examining cognitive engagement and self-regulated learning (SRL) using think-aloud protocols and log data from StoichTutor, an intelligent tutoring system for stoichiometry. Through a mixed-methods design, it analyzes utterance length, four SRL categories (Processing Information, Planning, Enacting, Realizing Errors), and SRL transitions via generalized linear models, logistic regressions, and ordered network analysis. Results show that gaming periods feature longer utterances and higher Processing Information and Realizing Errors, but lower Planning, with SRL transitions during gaming deviating from the canonical SRL cycle, suggesting maladaptive SRL or self-regulated non-learning. These findings imply that interventions should target underlying regulatory processes and scaffolding for planning and metacognitive monitoring, rather than solely discouraging gaming; the work also demonstrates the value of integrating think-aloud and log data for richer learning analytics.
Abstract
In digital learning systems, gaming the system refers to occasions when students attempt to succeed in an educational task by systematically taking advantage of system features rather than engaging meaningfully with the content. Often viewed as a form of behavioral disengagement, gaming the system is negatively associated with short- and long-term learning outcomes. However, little research has explored this phenomenon beyond its behavioral representation, leaving questions such as whether students are cognitively disengaged or whether they engage in different self-regulated learning (SRL) strategies when gaming largely unanswered. This study employs a mixed-methods approach to examine students' cognitive engagement and SRL processes during gaming versus non-gaming periods, using utterance length and SRL codes inferred from think-aloud protocols collected while students interacted with an intelligent tutoring system for chemistry. We found that gaming does not simply reflect a lack of cognitive effort; during gaming, students often produced longer utterances, were more likely to engage in processing information and realizing errors, but less likely to engage in planning, and exhibited reactive rather than proactive self-regulatory strategies. These findings provide empirical evidence supporting the interpretation that gaming may represent a maladaptive form of SRL. With this understanding, future work can address gaming and its negative impacts by designing systems that target maladaptive self-regulation to promote better learning.