High-frequency near-eye ground truth for event-based eye tracking

TL;DR

The paper addresses the lack of high-frequency, eye-level ground truth for event-based eye tracking by introducing a semi-automatic annotation pipeline that converts asynchronous event streams into 200 Hz RGB frames, detects eye movements, and estimates pupil centers using template matching and RANSAC, followed by human refinement. Applied to the Angelopoulos dataset, it provides 200 Hz pupil-center annotations along with blink and saccade labels, significantly enriching ground truth for training and evaluation. The approach reduces manual effort and enhances the reliability of pupil tracking in event-based systems, enabling more capable, low-power eye-tracking in smart eyewear. Overall, the work strengthens datasets in the emerging field of event-based eye tracking and supports faster development of real-time algorithms for near-eye devices.

Abstract

Event-based eye tracking is a promising solution for efficient and low-power eye tracking in smart eyewear technologies. However, the novelty of event-based sensors has resulted in a limited number of available datasets, particularly those with eye-level annotations, crucial for algorithm validation and deep-learning training. This paper addresses this gap by presenting an improved version of a popular event-based eye-tracking dataset. We introduce a semi-automatic annotation pipeline specifically designed for event-based data annotation. Additionally, we provide the scientific community with the computed annotations for pupil detection at 200Hz.

Figures (4)

• Figure 1: Examples of frames generated accumulating events at 200Hz
• Figure 2: Schema of the automatic annotation pipeline. The system takes the frame generated by accumulating events as input and first predicts if there has been eye movement. If a saccade is detected, the pupil center is determined using a template matching strategy followed by RANSAC estimation.
• Figure 3: Results from different operations performed during the pupil center localization. As we can see, the RANSAC estimation step increases the accuracy of the estimation made by the match template step.
• Figure 4: Interactive plot for annotation correction. Delta in x,y from an eye position to the next is shown in blue and red. Saccades are marked in violet (rising edge when the saccade starts, falling edge when the saccade ends). Blinks are indicated similarly in orange. At the end of the sequence, a possible anomaly is marked in green.