Investigating the Effect of Encumbrance on Gaze- and Touch-based Target Acquisition on Handheld Mobile Devices

TL;DR

This study addresses how carrying objects (encumbrance) affects gaze- and touch-based target acquisition on handheld devices. It compares gaze input via dwell time (with/without visual feedback), a multimodal GazeTouch approach, and traditional one- and two-handed touch postures in a two-condition walking task. The findings show that gaze with visual feedback remains stable under encumbrance, while touch-based interactions are more disrupted, and participants prefer gaze when encumbered and touch when unencumbered. The work contributes empirical evidence guiding modality selection in mobile UIs to account for situational impairments and suggests design refinements to improve gaze reliability and user experience in real-world contexts.

Abstract

The potential of using gaze as an input modality in the mobile context is growing. While users often encumber themselves by carrying objects and using mobile devices while walking, the impact of encumbrance on gaze input performance remains unexplored. To investigate this, we conducted a user study (N=24) to evaluate the effect of encumbrance on the performance of 1) Gaze using Dwell time (with/without visual feedback), 2) GazeTouch (with/without visual feedback), and 3) One- or two-hand touch input. While Touch generally performed better, Gaze, especially with feedback, showed a consistent performance regardless of whether participants were encumbered or unencumbered. Participants' preferences for input modalities varied with encumbrance: they preferred Gaze when encumbered, and touch when unencumbered. Our findings enhance understanding of the effect of encumbrance on gaze input and contribute towards selecting appropriate input modalities in future mobile user interfaces to account for situational impairments.

Figures (8)

• Figure 1: Participants walking path during the experiment. Similar to prior work 10.1145/2628363.2628382, we placed cones to guide the path that participants were required to traverse. Each cone was spaced approximately 85 cm apart, and the path was about one metre wide, as illustrated using the red arrows in the left image. Participants completed tasks in two conditions: encumbered by carrying bags and unencumbered. They were instructed to walk all the way to the end of the path, return via the designated return route, and continue walking additional laps until all required selections for the presented condition were completed.
• Figure 2: In each encumbrance level, participants had to complete nine selection tasks per input modality. In particular, they had to select the three targets at the top, three in the middle, and three in the bottom region. The target to be selected is highlighted in black (A). Visual feedback using red strokes surrounding the target the participants are looking at was provided in the Gaze (feedback) and GazeTouch (feedback) conditions (B). Once the correct target is selected, it changes its colour to green (C).
• Figure 3: Encumbrance significantly impacted the overall selection time regardless of the region for 1H-thumb and 2H-index, where the selection time was significantly slower when the participants were encumbered compared to when they were unencumbered. Selecting targets from the middle regions was significantly faster than the top regions when using 1H-thumb while encumbered. When unencumbered, selection time differed significantly across regions for 1H-thumb. The red and orange lines show the trend in encumbrance across regions. Error bars represent standard deviations.
• Figure 4: We calculated the percentage for successful selection of targets for each input modality while walking unencumbered and encumbered, as the completion rate. We found that encumbrance significantly reduced completion rates for GazeTouch, GazeTouch-F, and 1H-thumb input modalities. The error bars represent the standard deviations.
• Figure 5: We counted the number of times participants selected incorrect targets before selecting the correct one. While error counts varied across modalities due to encumbrance, we found no statistically significant differences. The error bars represent the standard deviation.