EyeTheia: A Lightweight and Accessible Eye-Tracking Toolbox

TL;DR

EyeTheia tackles the challenge of scalable, low-cost gaze tracking suitable for real-world cognitive and clinical research by marrying browser-based capture with a lightweight CNN backend inspired by iTracker and aided by MediaPipe landmark features. It evaluates two learning strategies—pretraining on mobile data and training from MPIIFaceGaze—plus a lightweight user calibration that consistently improves accuracy, enabling real-time, device-agnostic gaze estimation on consumer hardware. In a demanding Dot-Probe task, EyeTheia demonstrates strong left-right gaze agreement with a commercial tracker while exhibiting higher temporal jitter and slightly reduced spatial precision, underscoring its viability for coarse attentional metrics. The work delivers an open, transparent, and extensible toolkit, including code, trained models, and materials, poised to enhance reproducibility and accessibility in scalable gaze-related research.

Abstract

We introduce EyeTheia, a lightweight and open deep learning pipeline for webcam-based gaze estimation, designed for browser-based experimental platforms and real-world cognitive and clinical research. EyeTheia enables real-time gaze tracking using only a standard laptop webcam, combining MediaPipe-based landmark extraction with a convolutional neural network inspired by iTracker and optional user-specific fine-tuning. We investigate two complementary strategies: adapting a model pretrained on mobile data and training the same architecture from scratch on a desktop-oriented dataset. Validation results on MPIIFaceGaze show comparable performance between both approaches prior to calibration, while lightweight user-specific fine-tuning consistently reduces gaze prediction error. We further evaluate EyeTheia in a realistic Dot-Probe task and compare it to the commercial webcam-based tracker SeeSo SDK. Results indicate strong agreement in left-right gaze allocation during stimulus presentation, despite higher temporal variability. Overall, EyeTheia provides a transparent and extensible solution for low-cost gaze tracking, suitable for scalable and reproducible experimental and clinical studies. The code, trained models, and experimental materials are publicly available.

Figures (6)

• Figure 1: Overview of the user calibration pipeline used in both approaches. MediaPipe FaceMesh extracts facial landmarks, from which the iTracker inputs (eyes, face, face grid) are derived. The collected samples are used to fine-tune the model for the current user.
• Figure 2: Example of a single trial in the Dot-Probe Task used for evaluation. Each trial starts with a central fixation cross, followed by the presentation of two stimuli (negative vs. neutral). One stimulus is then replaced by a target dot, which the participant must localize via a key press while gaze is continuously recorded.
• Figure 3: Effect of the Smooth L1 (Huber) parameter $\beta$ on validation performance for Approach 2 trained on MPIIFaceGaze. (Left) Validation loss trajectories over training epochs for all tested values of $\beta$. (Right) Minimum validation loss achieved for each $\beta$, with the best performance obtained for $\beta = 0.8$.
• Figure 4: Comparison of gaze prediction errors before and after user-specific calibration for both EyeTheia approaches. Calibration consistently improves performance, highlighting its critical role for webcam-based eye tracking.
• Figure 5: ROI-based accuracy for EyeTheia (Approach 1) and SeeSo under different region-of-interest definitions.