Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types

Wolfgang Fuhl, Gjergji Kasneci, Enkelejda Kasneci

TL;DR

TEyeD introduces a public, large-scale dataset of over 20 million real-world eye images captured with seven head-mounted trackers, including VR/AR integrations. It provides unified 2D/3D landmarks and segmentation for pupil, iris, and eyelids, a 3D eyeball model, gaze vectors, and eye-movement types, across diverse tasks and conditions. The dataset enables cross-device generalization and robust gaze estimation research, supported by semi-supervised annotations (MAM) and baseline evaluations demonstrating improved cross-tracker performance with larger models. TEyeD is poised to impact computer vision, eye-tracking research, cognitive science, and VR/AR applications, despite licensing-driven omissions (e.g., Tobii data).

Abstract

We present TEyeD, the world's largest unified public data set of eye images taken with head-mounted devices. TEyeD was acquired with seven different head-mounted eye trackers. Among them, two eye trackers were integrated into virtual reality (VR) or augmented reality (AR) devices. The images in TEyeD were obtained from various tasks, including car rides, simulator rides, outdoor sports activities, and daily indoor activities. The data set includes 2D and 3D landmarks, semantic segmentation, 3D eyeball annotation and the gaze vector and eye movement types for all images. Landmarks and semantic segmentation are provided for the pupil, iris and eyelids. Video lengths vary from a few minutes to several hours. With more than 20 million carefully annotated images, TEyeD provides a unique, coherent resource and a valuable foundation for advancing research in the field of computer vision, eye tracking and gaze estimation in modern VR and AR applications. Download: https://es-cloud.cs.uni-tuebingen.de/d/8e2ab8c3fdd444e1a135/?p=%2FTEyeDS&mode=list

TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types

TL;DR

TEyeD introduces a public, large-scale dataset of over 20 million real-world eye images captured with seven head-mounted trackers, including VR/AR integrations. It provides unified 2D/3D landmarks and segmentation for pupil, iris, and eyelids, a 3D eyeball model, gaze vectors, and eye-movement types, across diverse tasks and conditions. The dataset enables cross-device generalization and robust gaze estimation research, supported by semi-supervised annotations (MAM) and baseline evaluations demonstrating improved cross-tracker performance with larger models. TEyeD is poised to impact computer vision, eye-tracking research, cognitive science, and VR/AR applications, despite licensing-driven omissions (e.g., Tobii data).

Abstract

We present TEyeD, the world's largest unified public data set of eye images taken with head-mounted devices. TEyeD was acquired with seven different head-mounted eye trackers. Among them, two eye trackers were integrated into virtual reality (VR) or augmented reality (AR) devices. The images in TEyeD were obtained from various tasks, including car rides, simulator rides, outdoor sports activities, and daily indoor activities. The data set includes 2D and 3D landmarks, semantic segmentation, 3D eyeball annotation and the gaze vector and eye movement types for all images. Landmarks and semantic segmentation are provided for the pupil, iris and eyelids. Video lengths vary from a few minutes to several hours. With more than 20 million carefully annotated images, TEyeD provides a unique, coherent resource and a valuable foundation for advancing research in the field of computer vision, eye tracking and gaze estimation in modern VR and AR applications. Download: https://es-cloud.cs.uni-tuebingen.de/d/8e2ab8c3fdd444e1a135/?p=%2FTEyeDS&mode=list

Paper Structure

This paper contains 7 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Comparison with Existing Data Sets
  4. Further Data Set Details
  5. Annotation Process
  6. Baseline Evaluations
  7. Conclusion

Figures (5)

  • Figure 1: Example images from our data set with annotations.
  • Figure 2: The logarithmic distribution of the pupil landmarks (left), iris landmarks (middle), and eyelid landmarks (right) in TüEyeD.
  • Figure 3: The area distribution for the pupil, iris, and eyelids on an $192\times144$ image resolution. The blue box corresponds to the 25th and 75th percentiles. Red crosses are the outlier and the red line corresponds to the median.
  • Figure 4: The logarithmic distribution of the gaze vector centered and mapped to a unit sphere in our data set.
  • Figure 5: The distributions for the eyeball center x and y as well as the distribution of the eyeball radius in our data set on a $192\times144$ image resolution. The blue box corresponds to the 25th and 75th percentiles. Red crosses are the outlier and the red line corresponds to the median.