Human Gaze and Head Rotation during Navigation, Exploration and Object Manipulation in Shared Environments with Robots

TL;DR

This work analyzes how humans gaze and move their heads when navigating, exploring, and manipulating objects in shared environments with a mobile robot, using the THÖR-MAGNI dataset. It develops metrics for gaze distribution, central bias, eye–head coordination, motion interplay, and semantic attention via YOLO-based object labeling, revealing task- and group-dependent patterns. Key findings include a robust central bias in fixations, variable eye–head contributions depending on action, and a notable shift of attention toward the robot across scenarios. The results support deploying gaze-informed perception and interaction models in robots, highlighting the robot’s presence as a strong driver of human attention in shared spaces.

Abstract

The human gaze is an important cue to signal intention, attention, distraction, and the regions of interest in the immediate surroundings. Gaze tracking can transform how robots perceive, understand, and react to people, enabling new modes of robot control, interaction, and collaboration. In this paper, we use gaze tracking data from a rich dataset of human motion (THÖR-MAGNI) to investigate the coordination between gaze direction and head rotation of humans engaged in various indoor activities involving navigation, interaction with objects, and collaboration with a mobile robot. In particular, we study the spread and central bias of fixations in diverse activities and examine the correlation between gaze direction and head rotation. We introduce various human motion metrics to enhance the understanding of gaze behavior in dynamic interactions. Finally, we apply semantic object labeling to decompose the gaze distribution into activity-relevant regions.

Figures (5)

• Figure 1: A participant of the THÖR-MAGNI dataset attends to instructions of the mobile robot schreiter2023advantages. Top: Illustration of the visual difference between the head orientation (red) and gaze direction (green). Bottom: a sequence of gazes on the mobile robot, followed by a shift of attention to the goal point that the robot cued. This shift is followed by a head rotation to center the visual field on the goal point. Fixations are shown with white circles, and their sequences are connected by red lines.
• Figure 2: Fixation locations in the THÖR-MAGNI dataset. Ellipses represent areas containing $25\%$, $50\%$, $80\%$, and $90\%$ (Black Labels) of all recorded gazes. Blue Labels indicate the percentage of the 1920x1080 image included in each ellipse.
• Figure 3: Heatmaps of fixation locations in the dataset with average Fixation duration (FD) and overall Fixation rate (FR) per role and the comprised micro-actions. Visual axes are shown with dotted lines. Central coordinates and the spread of the central biases are listed in Tables \ref{['tab:activ-carry']} and \ref{['tab:activ-visit']}.
• Figure 4: Showing how much the head direction contributes to the total gaze direction, depending on the eccentricity of the gaze angle with respect to the body orientation. This figure applies to movements where the head and eyes move horizontally in the same direction.
• Figure 5: Distribution of fixations on objects by participants in Scenarios 2, 3A, and 3B (left) and 1A and 1B (right).