Bumpy Ride? Understanding the Effects of External Forces on Spatial Interactions in Moving Vehicles

TL;DR

This study addresses how external vehicle forces affect spatial interactions with AR/VR head-mounted displays during movement. It employs a field-based Fitts' Law task comparing four input modalities (Gaze&Pinch, DirectTouch, Handray, HeadGaze) across standstill and movement, with road-type and curve-type labels, and collects objective and subjective metrics. Key findings show movement degrades throughput, increases error rate and workload, and alters trajectories differently across interaction methods, with Handray emerging as the most robust option for movement. The work contributes a labeled dataset linking road/curve conditions to interaction performance, quantitative and qualitative insights into usability and workload, and practical guidelines for selecting and refining input methods in moving-vehicle AR/VR contexts. It highlights the importance of motion fidelity considerations and proposes directions for adaptive interaction techniques and future research in in-vehicle HMD usability.

Abstract

As the use of Head-Mounted Displays in moving vehicles increases, passengers can immerse themselves in visual experiences independent of their physical environment. However, interaction methods are susceptible to physical motion, leading to input errors and reduced task performance. This work investigates the impact of G-forces, vibrations, and unpredictable maneuvers on 3D interaction methods. We conducted a field study with 24 participants in both stationary and moving vehicles to examine the effects of vehicle motion on four interaction methods: (1) Gaze&Pinch, (2) DirectTouch, (3) Handray, and (4) HeadGaze. Participants performed selections in a Fitts' Law task. Our findings reveal a significant effect of vehicle motion on interaction accuracy and duration across the tested combinations of Interaction Method x Road Type x Curve Type. We found a significant impact of movement on throughput, error rate, and perceived workload. Finally, we propose future research considerations and recommendations on interaction methods during vehicle movement.

Figures (18)

• Figure 1: Route, instructions, and starting positions for all 24 participants. 30km/h is the standard speed employed on the course.
• Figure 2: Fitts' Law Task as observed through the Varjo XR-3 with passthrough enabled. The cursor of each interaction method is shown in green and highlights targets in red when hovering, indicating the ability to select the target. One of the seven targets is always highlighted in blue to indicate it should be selected next, until a successful selection is performed.
• Figure 3: Vehicle acceleration on the three road conditions: SmoothRoad, MixedRoad, and BumpyRoad. Recordings from P08 during Gaze&Pinch. The x-axis describes longitudinal, the y-axis lateral, and the z-axis vertical acceleration.
• Figure 4: Overview of road types investigated in our study, depicting variations in surface quality.
• Figure 5: Interaction effect on NASA-rTLX: Total Score