GAIPAT -Dataset on Human Gaze and Actions for Intent Prediction in Assembly Tasks
Maxence Grand, Damien Pellier, Francis Jambon
TL;DR
GAIPAT addresses how gaze patterns relate to actions in assembly tasks by providing a richly annotated dataset collected from approximately 80 participants using both remote and head-mounted eye trackers across sitting and standing postures. The Methods outline a detailed experimental setup, data collection pipeline, and two-tier annotations (atomic events and slide AOIs) to enable precise linking of gaze with hand actions. The dataset is organized into Setup Data and Participant Data, with extensive files describing blocks, instructions, slides, participant configurations, and per-trial gaze and gesture data, while preserving privacy through controlled video usage. This resource enables human intent prediction, benchmarking of gaze-based cobot control strategies, and safe, efficient human-robot collaboration in industrial-like assembly tasks. The work thus provides a practical, extensible platform for studying attentional dynamics and action anticipation in shared-work environments.
Abstract
The primary objective of the dataset is to provide a better understanding of the coupling between human actions and gaze in a shared working environment with a cobot, with the aim of signifcantly enhancing the effciency and safety of humancobot interactions. More broadly, by linking gaze patterns with physical actions, the dataset offers valuable insights into cognitive processes and attention dynamics in the context of assembly tasks. The proposed dataset contains gaze and action data from approximately 80 participants, recorded during simulated industrial assembly tasks. The tasks were simulated using controlled scenarios in which participants manipulated educational building blocks. Gaze data was collected using two different eye-tracking setups -head-mounted and remote-while participants worked in two positions: sitting and standing.