Enhancement of Co-located Shared VR Experiences: Representing Non-HMD Observers on Both HMD and 2D Screen

TL;DR

This study investigates improving co-located VR experiences by visually representing non-HMD observers on both HMDs and 2D screens. It systematically compares five conditions, including No Representation, Virtual Avatar, Passthrough Observer, Passthrough Small Window, and Passthrough Augmented Reality, in a two-person setup (one HMD user, one observer) performing a VR game task with a secondary N-back task. Results show that observer representations generally benefit both HMD users and observers, with Passthrough-based methods enhancing information flow and communication while trade-offs in immersion occur; PO (Passthrough Observer) often yields the best balance for HMD users, and observers show mixed preferences between realism and avatars. The authors propose actionable design guidelines tailored to HMD and 2D screen displays and discuss limitations such as single-observer conditions and latency, outlining directions for broader, more scalable future work. Overall, the work offers practical insights to enhance shared VR experiences by integrating observers more effectively into both head-mounted and traditional display modalities.

Abstract

Virtual reality (VR) not only allows head-mounted display (HMD) users to immerse themselves in virtual worlds but also to share them with others. When designed correctly, this shared experience can be enjoyable. However, in typical scenarios, HMD users are isolated by their devices, and non-HMD observers lack connection with the virtual world. To address this, our research investigates visually representing observers on both HMD and 2D screens to enhance shared experiences. The study, including five representation conditions, reveals that incorporating observer representation positively impacts both HMD users and observers. For how to design and represent them, our work shows that HMD users prefer methods displaying real-world visuals, while observers exhibit diverse preferences regarding being represented with real or virtual images. We provide design guidelines tailored to both displays, offering valuable insights to enhance co-located shared VR experiences for HMD users and non-HMD observers.

Figures (4)

• Figure 1: Living room-like environment with the TV, HMD user, and non-HMD observer positions during the experiment.
• Figure 2: System usability ratings for (a) HMD users and (b) non-HMD observers in each condition. Error bars indicate 95% confidence intervals. *, **, and *** indicate statistical significance at the $p < 0.05, p < 0.01,$ and $p < 0.001$ levels, respectively.
• Figure 3: (a) Sense of Being There, Involvement, and Co-presence ratings for HMD users in each condition. (b) Sense of Being There ratings for non-HMD observers in each condition. Error bars indicate 95% confidence intervals. *, **, and *** indicate statistical significance at the $p < 0.05, p < 0.01,$ and $p < 0.001$ levels, respectively.
• Figure 4: SAM ratings for (a) HMD users and (b) non-HMD observers in each condition. Error bars indicate 95% confidence intervals. *, **, and *** indicate statistical significance at the $p < 0.05, p < 0.01,$ and $p < 0.001$ levels, respectively.