A Hands-free Spatial Selection and Interaction Technique using Gaze and Blink Input with Blink Prediction for Extended Reality
Tim Rolff, Jenny Gabel, Lauren Zerbin, Niklas Hypki, Susanne Schmidt, Markus Lappe, Frank Steinicke
TL;DR
The paper introduces Gaze+Blink, a hands-free interaction technique for spatial UIs that combines gaze, intentional blinks, and, when needed, head movements to enable discrete selections and continuous scrolling/dragging. Through two user studies, the authors show that Gaze+Blink achieves comparable task completion times to Gaze+Pinch but with higher selection error rates, motivating the development of BlinkPlus, a deep-learning based filter to distinguish voluntary from involuntary blinks. BlinkPlus partially mitigates errors and demonstrates that blink-based interactions can be viable alternatives in XR, particularly for constrained or privacy-sensitive contexts, while revealing challenges such as involuntary blinks, eye-tracking reliability, and user variability. The work contributes a complete interaction model, a DL-based blink classifier, and a rigorous comparative evaluation, offering a foundation for future standardized assessments and accessibility-focused improvements in blink-based XR interactions.
Abstract
Gaze-based interaction techniques have created significant interest in the field of spatial interaction. Many of these methods require additional input modalities, such as hand gestures (e.g., gaze coupled with pinch). Those can be uncomfortable and difficult to perform in public or limited spaces, and pose challenges for users who are unable to execute pinch gestures. To address these aspects, we propose a novel, hands-free Gaze+Blink interaction technique that leverages the user's gaze and intentional eye blinks. This technique enables users to perform selections by executing intentional blinks. It facilitates continuous interactions, such as scrolling or drag-and-drop, through eye blinks coupled with head movements. So far, this concept has not been explored for hands-free spatial interaction techniques. We evaluated the performance and user experience (UX) of our Gaze+Blink method with two user studies and compared it with Gaze+Pinch in a realistic user interface setup featuring common menu interaction tasks. Study 1 demonstrated that while Gaze+Blink achieved comparable selection speeds, it was prone to accidental selections resulting from unintentional blinks. In Study 2 we explored an enhanced technique employing a deep learning algorithms for filtering out unintentional blinks.