Understanding Physical Breakdowns in Virtual Reality

TL;DR

This work investigates physical breakdowns in VR—abrupt disruptions caused by collisions with the real environment in uncontrolled spaces—through an attentional-model lens. It combines empirical studies of safety-boundary breakouts, a finger-based motion technique called FingerMapper to reduce physical interaction while preserving presence, and speculative design work on perceptual manipulation (VPPMs) that could be exploited to induce harm. Key contributions include empirical insights into why users breach safety boundaries, a validated mitigation approach that maintains presence with fewer physical movements, and a framework for recognizing and countering perceptual-attacks in XR. The research emphasizes the need for safety-focused metrics and mechanisms to ensure safer everyday VR experiences and informs future design and policy for VR/XR safety.

Abstract

Virtual Reality (VR) moves away from well-controlled laboratory environments into public and personal spaces. As users are visually disconnected from the physical environment, interacting in an uncontrolled space frequently leads to collisions and raises safety concerns. In my thesis, I investigate this phenomenon which I define as the physical breakdown in VR. The goal is to understand the reasons for physical breakdowns, provide solutions, and explore future mechanisms that could perpetuate safety risks. First, I explored the reasons for physical breakdowns by investigating how people interact with the current VR safety mechanism (e.g., Oculus Guardian). Results show one reason for breaking out of the safety boundary is when interacting with large motions (e.g., swinging arms), the user does not have enough time to react although they see the safety boundary. I proposed a solution, FingerMapper, that maps small-scale finger motions onto virtual arms and hands to enable whole-body virtual arm motions in VR to avoid physical breakdowns. To demonstrate future safety risks, I explored the malicious use of perceptual manipulations (e.g., redirection techniques) in VR, which could deliberately create physical breakdowns without users noticing. Results indicate further open challenges about the cognitive process of how users comprehend their physical environment when they are blindfolded in VR.

Figures (4)

• Figure 1: My research explores physical breakdowns in VR within a structure adapted from the attentional model for synthetic environment draperTelepresence1998, in which I position my research projects (sky-blue) and future directions (pink).
• Figure 2: Despite having the safety boundary, VR users may still break out accidentally or intentionally (the red circle), leading to safety concerns (e.g., collision). I implemented an apparatus (FruitSlicer) to replicate the break-out experience.
• Figure 3: FingerMapper leverages less physically demanding finger motions, enabling whole-body virtual arm movements in confined spaces (e.g., the passenger seat of a car) with fewer collisions while preserving presence and enjoyment.
• Figure 4: (a) Participants created scenarios in the speculative design workshop to speculate on the malicious use of VPPMs. The left shows the puppetry attack that directs a user to fall over the stairs. The right illustrates the mismatching attack in that the VR user punches a virtual zombie overlay on a bystander. (b) Physical harm was identified in all collected scenarios, from mild pain or discomfort to extreme cases like death.