Effects of Virtual Controller Representation and Virtuality on Selection Performance in Extended Reality

TL;DR

The study investigates how visual representations of input controllers influence target selection performance in VR and MR using a Fitts' law–based ISO-9241-411 task across four representation conditions. It employs a 2×4 within-subjects design (VR vs MR; Controller, Hand, Both, None) with targets at three amplitudes and four widths, totaling 12 IDs and 40 participants; performance metrics include mean movement time ($MT$), effective throughput ($TP_e$), depth deviation, and error rate, alongside subjective feedback. Key findings show that XR mode (VR vs MR) yields no significant performance differences, while controller representations significantly affect MT, TP_e, and depth deviation, with None performing worst. Subjective results indicate MR alters perceived performance and preferences differently from VR, underscoring the need for MR-specific design considerations, especially for spatial awareness and latency perceptions. Overall, the work suggests that performance is largely transferable between VR and MR, but interface design should tailor visual representations and depth cues to MR's perceptual context for optimal user experience and safety.

Abstract

We present an experiment exploring how the controller's virtual representation impacts target acquisition performance across MR and VR contexts. Participants performed selection tasks comparing four visual configurations: a virtual controller, a virtual hand, both the controller and the hand, and neither representation. We found performance comparable between VR and MR, and switching between them did not impact the user's ability to perform basic tasks. Controller representations mimicking reality enhanced performance across both modes. However, users perceived performance differently in MR, indicating the need for unique MR design considerations, particularly regarding spatial awareness.

Figures (7)

• Figure 1: The Fitts’ Law experiment, at the start and mid points of an ID, in the MR and the VR conditions. The green target indicates the participant can rest to avoid fatigue.
• Figure 2: The real-world experiment location visible in MR and the VR recreation of the location.
• Figure 3: Fitts' law analysis by XR mode and controller modes.
• Figure 4: Error rate across controller and XR modes. Error bars indicate a 95% confidence interval.
• Figure 5: Selection Time, effective throughput, and depth deviation (Delta Z) across controller and XR mode. Error bars indicate a 95% confidence interval. Significance is indicated at each level: $<$ 0.001 = ***, $<$ 0.01 = **, $<$ 0.05 = *.