Just stop doing everything for now!: Understanding security attacks in remote collaborative mixed reality

TL;DR

This paper tackles the security risks of remote collaborative MR by implementing four MR-specific attacks (latency, click redirection, object occlusion, spatial occlusion) on a HoloLens 2-based prototype and conducting a two-stage user study. It finds that users have low recognition of immersive MR threats and often misattribute issues to glitches or partner error, with mitigations not always aligning with actual actions. The work offers design recommendations—improved object visibility, immersive security cues, and targeted user training—to strengthen security and trust in remote MR collaboration. Overall, the findings underscore the need for user-centered security enhancements as MR-enabled remote collaboration becomes more widespread.

Abstract

Mixed Reality (MR) devices are being increasingly adopted across a wide range of real-world applications, ranging from education and healthcare to remote work and entertainment. However, the unique immersive features of MR devices, such as 3D spatial interactions and the encapsulation of virtual objects by invisible elements, introduce new vulnerabilities leading to interaction obstruction and misdirection. We implemented latency, click redirection, object occlusion, and spatial occlusion attacks within a remote collaborative MR platform using the Microsoft HoloLens 2 and evaluated user behavior and mitigations through a user study. We compared responses to MR-specific attacks, which exploit the unique characteristics of remote collaborative immersive environments, and traditional security attacks implemented in MR. Our findings indicate that users generally exhibit lower recognition rates for immersive attacks (e.g., spatial occlusion) compared to attacks inspired by traditional ones (e.g., click redirection). Our results demonstrate a clear gap in user awareness and responses when collaborating remotely in MR environments. Our findings emphasize the importance of training users to recognize potential threats and enhanced security measures to maintain trust in remote collaborative MR systems.

Figures (4)

• Figure 1: (a) The target puzzle solution and provided shapes. (b) Participants place the 2D cutouts within their portal to create and share as a 3D virtual object. (c) Partially completed tower in the MR environment.
• Figure 2: (a) Average time taken by participants across conditions. (b) Participants' average ratings of performance, showing higher ratings for spatial occlusion despite worse performance. (c) Responses to security concerns from the MRC Questionnaire mapped by attack condition.
• Figure 3: This graph shows the distribution of types of attacks recognized for each condition. There is low recognition and high misclassification by the participants for the implemented attacks.
• Figure 4: Distribution of mitigation techniques reported and performed for different attack conditions. There is a discrepancy in the mitigation techniques reported and performed by the participants for both the spatial occlusion and click redirection attacks.