The People's Gaze: Co-Designing and Refining Gaze Gestures with General Users and Gaze Interaction Experts

TL;DR

A two-phase methodology for developing intuitive gaze gestures is introduced and it is found that non-experts, after a brief introduction, intuitively anchor gestures in familiar metaphors and develop a compositional grammar to ensure intentionality and mitigate the classic Midas Touch problem.

Abstract

As eye-tracking becomes increasingly common in modern mobile devices, the potential for hands-free, gaze-based interaction grows, but current gesture sets are largely expert-designed and often misaligned with how users naturally move their eyes. To address this gap, we introduce a two-phase methodology for developing intuitive gaze gestures. First, four co-design workshops with 20 non-expert participants generated 102 initial concepts. Next, four gaze interaction experts reviewed and refined these into a set of 32 gestures. We found that non-experts, after a brief introduction, intuitively anchor gestures in familiar metaphors and develop a compositional grammar; i.e., activation (dwell) + action (gaze gesture or blink), to ensure intentionality and mitigate the classic Midas Touch problem. Experts prioritized gestures that are ergonomically sound, aligned with natural saccades, and reliably distinguishable. The resulting user-grounded, expert-validated gesture set, along with actionable design principles, provides a foundation for developing intuitive, hands-free interfaces for gaze-enabled devices.

Figures (9)

• Figure 1: The structured four-step procedure used in the Phase 1 co-design workshops. Each session included a 10-minute introduction, 20 minutes for group ideation and sketching, 20 minutes for presentation and reflection, and a final 10-minute peer evaluation where groups assessed each other's designs.
• Figure 2: The complete set of 59 unique gestures generated by non-expert participants during Phase 1, shown in their standardized format. After correcting for duplicate labels (merging original gestures $\beta$-9 and $\beta$-12 into $\beta$-8 and $\beta$-11, respectively), the set was divided for peer evaluation into Group $\alpha$ (35 gestures) and Group $\beta$ (24 gestures).Full textual descriptions of each gesture, including timing and command mapping, are provided in the appendix \ref{['app:gesture_icon_definition']}.
• Figure 3: The user-generated gesture set from Phase 1, categorized by intended command function. This view highlights design patterns for discrete actions (e.g., Open, Exit) and continuous actions (e.g., Zoom, Light down/up). Gestures from Group $\alpha$ are denoted by red borders, and those from Group $\beta$ by blue borders. The bottom-right $\alpha$-26 gesture represents a user-defined design that does not belong to any predefined command category.
• Figure 4: The gesture consolidation process during the expert review discussion. Functionally and visually similar designs, such as these horizontal swipes, were merged into a single representative gesture to create a non-redundant set.
• Figure 5: The final expert-validated gesture set, containing 32 robust designs refined from the initial user-generated concepts. This set is composed of 15 gestures from Group $\alpha$ and 17 from Group $\beta$, representing designs that experts deemed ergonomically sound, semantically clear, and technically viable.