Implicit gaze research for XR systems

TL;DR

The paper addresses the underutilized potential of eye tracking in XR to infer user intent and cognitive states beyond explicit input. It defines explicit and implicit gaze modes, surveys current applications, and outlines research directions to harness implicit gaze. Key contributions include outlining how implicit gaze can predict timing and targets, enhance interaction primitives, detect errors, quantify cognitive state with gaze metrics, and optimize UIs based on context prior to interaction, along with mode-transition recommendations and privacy considerations. It argues that implicit gaze can enable contextual, proactive interfaces with reduced cognitive and physical load, and emphasizes the need for large-scale naturalistic gaze data, context sensing, and interpretable AI to realize these gains.

Abstract

Although eye-tracking technology is being integrated into more VR and MR headsets, the true potential of eye tracking in enhancing user interactions within XR settings remains relatively untapped. Presently, one of the most prevalent gaze applications in XR is input control; for example, using gaze to control a cursor for pointing. However, our eyes evolved primarily for sensory input and understanding of the world around us, and yet few XR applications have leveraged natural gaze behavior to infer and support users' intent and cognitive states. Systems that can represent a user's context and interaction intent can better support the user by generating contextually relevant content, by making the user interface easier to use, by highlighting potential errors, and more. This mode of application is not fully taken advantage of in current commercially available XR systems and yet it is likely where we'll find paradigm-shifting use cases for eye tracking. In this paper, we elucidate the state-of-the-art applications for eye tracking and propose new research directions to harness its potential fully.