Beyond Instructed Tasks: Recognizing In-the-Wild Reading Behaviors in the Classroom Using Eye Tracking

TL;DR

This work addresses the ecological validity gap in reading-behavior recognition by comparing instructed versus in-the-wild reading in a classroom using eye-tracking. It develops a mixed-method framework that combines a human-driven taxonomy based on velocity, density, and sequentiality with a lightweight $2$D CNN classifier to recognize in-the-wild reading behaviors, achieving a Macro $F_1$ of $0.80$ in LOPOCV. Key contributions include publicly available eye-tracking datasets for both contexts, a taxonomy of behaviors (e.g., static, deep, sequential, non-sequential, skimming, previewing/mapping), and a real-time classification pipeline suitable for classroom use. The findings highlight richer, more diverse reading behaviors in naturalistic settings and demonstrate the potential for real-time, targeted instructional feedback, while acknowledging limitations like sample size and material variety that call for broader future work.

Abstract

Understanding reader behaviors such as skimming, deep reading, and scanning is essential for improving educational instruction. While prior eye-tracking studies have trained models to recognize reading behaviors, they often rely on instructed reading tasks, which can alter natural behaviors and limit the applicability of these findings to in-the-wild settings. Additionally, there is a lack of clear definitions for reading behavior archetypes in the literature. We conducted a classroom study to address these issues by collecting instructed and in-the-wild reading data. We developed a mixed-method framework, including a human-driven theoretical model, statistical analyses, and an AI classifier, to differentiate reading behaviors based on their velocity, density, and sequentiality. Our lightweight 2D CNN achieved an F1 score of 0.8 for behavior recognition, providing a robust approach for understanding in-the-wild reading. This work advances our ability to provide detailed behavioral insights to educators, supporting more targeted and effective assessment and instruction.

Figures (13)

• Figure 1: Study Structure: The study is organized into three main sections: setup, instructed behavior activity, and in-the-wild reading. During the setup phase, students completed an eye-tracking calibration and received instructions. In the instructed behavior activity, students were directed to read in a specified manner. For the in-the-wild reading section, students were asked to read a passage as they would naturally in a classroom setting, without access to any questions. After completing the reading, they moved on to the question segment, where they could access both the questions and the text. However, they could not revisit previous questions once they had submitted an answer for each one.
• Figure 2: Data Collection Protocol: Deployed using consumer-level HP laptops to present the custom web application and a Tobii Pro Spark to collect eye-tracking data. The web application provides a digital PDF reader with a side panel for questions.
• Figure 3: Page AOI Encoding: Illustration of the relationship between screen coordinate space (SCS) and an example page coordinate space (PCS) of the PDF viewer within the custom web application.
• Figure 4: Example gaze scanpath: Utilizing a blue-to-red gradient to illustrate the temporal evaluation of the scanpath. The visualization is overlaid on top of the text to help contextualize the gaze. The beginning of the scanpath is painted blue $(0,0,255)$, throughout time the blue is replaced with red, and at the end is marked with full red color $(255,0,0)$.
• Figure 5: Example Human-Labeled In-the-Wild Behaviors: Gaze scanpaths from participants 1 (top), 15 (center), and 47 (bottom). illustrating how different reading behaviors are leveraged to explore and comprehend the text.