Immersive In Situ Visualizations for Monitoring Architectural-Scale Multiuser MR Experiences

TL;DR

Architectural-scale Co-MUMR enables multiuser MR experiences in large spaces, but hosts lack real-time visibility into visitor engagement, device performance, and events. The authors develop immersive in situ visualizations to surface I1–I3 (visitor engagement, device performance, real-time events) and implement them in a host-oriented system using Unity Netcode on Quest devices. They present four hosting experiences (E1–E4) to motivate design, propose adaptive, subject- and host-centric visualization configurations, and implement a comprehensive visualization suite (visitor views, bounding-box cues, network curves, and calibration indicators) with a data-flow and VFX-based rendering approach. They discuss challenges such as visual clutter and suggest future work on prioritization and deeper analysis of engagement plus an evaluation plan via a puppet exhibition. The approach offers a practical path to improving hosting quality and visitor experience in large-scale MR installations.

Abstract

Mixed reality (MR) environments provide great value in displaying 3D virtual content. Systems facilitating co-located multiuser MR (Co-MUMR) experiences allow multiple users to co-present in a shared immersive virtual environment with natural locomotion. They can be used to support a broad spectrum of applications such as immersive presentations, public exhibitions, psychological experiments, etc. However, based on our experiences in delivering Co-MUMR experiences in large architectures and our reflections, we noticed that the crucial challenge for hosts to ensure the quality of experience is their lack of insight into the real-time information regarding visitor engagement, device performance, and system events. This work facilitates the display of such information by introducing immersive in situ visualizations.

Figures (6)

• Figure 1: Exemplary immersive in situ visualizations. (A) A sketch of a collection of in situ visualizations helping hosts access real-time information regarding the multiuser MR experience. (B) Visitor engagement visualization displays their rendered view by side. (C) Embedded visualization of real-time device metrics, including rendering framerate and battery status. (D) Visualization of past calibration events with a color-coded circular-shaped chart around a calibration station.
• Figure 2: Photos and screenshots of our previously organized and hosted Co-MUMR experiences.
• Figure 3: The screenshot of the proposed visitor engagement visualizations for displaying visitors' detailed rendered views placed in space embedded by the side of visitors (A), projected in front of the host (B and C), and a coarse view of visitors' perspective with view frustums (D). Spatial locations of individual visitors with embedded giant arrows (E), visual links from the host to visitors (F and G), and a coarse view of the spatial locations of a group of visitors (H).
• Figure 4: Screenshots of the visualizations for showing detailed performance metrics with information panel placed by side of individual HMDs (A), projected in front of the host (B and C). Coarse overview of the performance by encoding into the color of visitor bounding boxes (D) and visual links between visitors and the host (E). Hand tracking performance in (F) and networking performance in (G). Device performance history alongside its spatial movement trajectory in (H).
• Figure 5: Visualizations for showing system events: offline events (A) and calibration events (B).