Gaze-Hand Steering for Travel and Multitasking in Virtual Environments

TL;DR

This paper addresses gaze-dominant VR navigation under multitasking by introducing gaze-hand steering, which couples eye-gaze direction with hand-pointing to define travel while preserving free look. The method uses either a joystick or a waist-level speed circle for speed control and enables concurrent tasks such as object selection during locomotion. A within-subject study (N=20) compares the approach to the Magic Carpet baseline, reporting strong immersion, moderate workload, and robust multitasking performance with fewer collisions. The findings demonstrate that gaze-hand steering maintains navigation performance while enhancing comfort and spatial awareness, supporting its applicability to gaze-dominant VR applications requiring precise control amid multitasking.

Abstract

As head-mounted displays (HMDs) with eye-tracking become increasingly accessible, the need for effective gaze-based interfaces in virtual reality (VR) grows. Traditional gaze- or hand-based navigation often limits user precision or impairs free viewing, making multitasking difficult. We present a gaze-hand steering technique that combines eye-tracking with hand-pointing: users steer only when gaze aligns with a hand-defined target, reducing unintended actions and enabling free look. Speed is controlled via either a joystick or a waist-level speed circle. We evaluated our method in a user study (N=20) across multitasking and single-task scenarios, comparing it to a similar technique. Results show that gaze-hand steering maintains performance and enhances user comfort and spatial awareness during multitasking. Our findings support the use of gaze-hand steering in gaze-dominant VR applications requiring precision and simultaneous interaction. Our method significantly improves VR navigation in gaze-dominant, multitasking-intensive applications, supporting immersion and efficient control.

Figures (4)

• Figure 1: Perspective of the virtual environment.
• Figure 2: Path used in the experiment (350 m). Yellow dots indicate the rings placement.
• Figure 3: Comparison of performance metrics across joystick and speed circle techniques, including ring navigation and target shooting tasks, to evaluate multitasking performance during VR navigation: (a) percentage of time in motion (Flying vs. Idle), (b) total path length (units), (c) average navigation speed (m/s), and (d) total collision time (seconds).
• Figure 4: Comparing each of our conditions with the Magic Carpet's three conditions.