See Where You Read with Eye Gaze Tracking and Large Language Model
Sikai Yang, Gang Yan, Wan Du
TL;DR
RT^2H addresses the challenge of tracking reading progress when gaze accuracy is coarse relative to line spacing. It combines linear-reading constraints with jump-reading relocation, aided by punctuation anchors and LLM-assisted candidate election. Key contributions include real-time reading tracking with dynamic line-gaze calibration and 84% jump-reading relocation accuracy, demonstrated in controlled experiments, plus field testing showing a 13.5% reduction in reading time and positive user feedback. The work suggests practical, field-ready reading-tracking highlighting enabled by a synergy of gaze analysis and language perception.
Abstract
Losing track of reading progress during line switching can be frustrating. Eye gaze tracking technology offers a potential solution by highlighting read paragraphs, aiding users in avoiding wrong line switches. However, the gap between gaze tracking accuracy (2-3 cm) and text line spacing (3-5 mm) makes direct application impractical. Existing methods leverage the linear reading pattern but fail during jump reading. This paper presents a reading tracking and highlighting system that supports both linear and jump reading. Based on experimental insights from the gaze nature study of 16 users, two gaze error models are designed to enable both jump reading detection and relocation. The system further leverages the large language model's contextual perception capability in aiding reading tracking. A reading tracking domain-specific line-gaze alignment opportunity is also exploited to enable dynamic and frequent calibration of the gaze results. Controlled experiments demonstrate reliable linear reading tracking, as well as 84% accuracy in tracking jump reading. Furthermore, real field tests with 18 volunteers demonstrated the system's effectiveness in tracking and highlighting read paragraphs, improving reading efficiency, and enhancing user experience.