That Doesn't Go There: Attacks on Shared State in Multi-User Augmented Reality Applications

TL;DR

This work demonstrates a series of novel attacks on multiple AR frameworks with shared states, focusing on three publicly-accessible frameworks, showing that these frameworks, while using different underlying implementations, scopes, and mechanisms to read from and write to the shared state, have shared vulnerability to a unified threat model.

Abstract

Augmented Reality (AR) is expected to become a pervasive component in enabling shared virtual experiences. In order to facilitate collaboration among multiple users, it is crucial for multi-user AR applications to establish a consensus on the "shared state" of the virtual world and its augmentations, through which they interact within augmented reality spaces. Current methods to create and access shared state collect sensor data from devices (e.g., camera images), process them, and integrate them into the shared state. However, this process introduces new vulnerabilities and opportunities for attacks. Maliciously writing false data to "poison" the shared state is a major concern for the security of the downstream victims that depend on it. Another type of vulnerability arises when reading the shared state; by providing false inputs, an attacker can view hologram augmentations at locations they are not allowed to access. In this work, we demonstrate a series of novel attacks on multiple AR frameworks with shared states, focusing on three publicly-accessible frameworks. We show that these frameworks, while using different underlying implementations, scopes, and mechanisms to read from and write to the shared state, have shared vulnerability to a unified threat model. Our evaluation of these state-of-art AR applications demonstrates reliable attacks both on updating and accessing shared state across the different systems. To defend against such threats, we discuss a number of potential mitigation strategies that can help enhance the security of multi-user AR applications.

Figures (16)

• Figure 1: AR processing pipeline. An AR device senses the environment, processes the sensed data, and uploads information to the shared state. The shared state returns an augmentation overlaid onto the user's display.
• Figure 2: Attacks on AR shared state. Read attack: A private hologram is read outside the area it was written to (beach instead of office). Write attack: A hologram is written to an area where the attacker is not present (pipes instead of field).
• Figure 3: Remote read attack in Scenario A. Left: A victim places a hologram in front of a yellow sign. Right: An attacker is able to view the hologram from a photograph without being physically near the yellow sign.
• Figure 4: Remote write attack in scenario A. Left: An attacker is able to write a hologram at a real-world location (a desk) without being physically present. Right: A victim views the unexpected hologram on the desk.
• Figure 5: Triggered remote write attack in scenario A. Left: An attacker employs triggered features to remotely write a hologram at a real-world location without being physically present. Right: A victim encounters an unexpected hologram on their desk, triggered by features injected by the attacker.