Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Remote Keylogging Attacks in Multi-user VR Applications

Zihao Su, Kunlin Cai, Reuben Beeler, Lukas Dresel, Allan Garcia, Ilya Grishchenko, Yuan Tian, Christopher Kruegel, Giovanni Vigna

TL;DR

This study proposes a remote attack that utilizes the avatar rendering information collected from an adversary's game clients to extract user-typed secrets like credit card information, passwords, or private conversations from multi-user VR applications.

Abstract

As Virtual Reality (VR) applications grow in popularity, they have bridged distances and brought users closer together. However, with this growth, there have been increasing concerns about security and privacy, especially related to the motion data used to create immersive experiences. In this study, we highlight a significant security threat in multi-user VR applications, which are applications that allow multiple users to interact with each other in the same virtual space. Specifically, we propose a remote attack that utilizes the avatar rendering information collected from an adversary's game clients to extract user-typed secrets like credit card information, passwords, or private conversations. We do this by (1) extracting motion data from network packets, and (2) mapping motion data to keystroke entries. We conducted a user study to verify the attack's effectiveness, in which our attack successfully inferred 97.62% of the keystrokes. Besides, we performed an additional experiment to underline that our attack is practical, confirming its effectiveness even when (1) there are multiple users in a room, and (2) the attacker cannot see the victims. Moreover, we replicated our proposed attack on four applications to demonstrate the generalizability of the attack. Lastly, we proposed a defense against the attack, which has been implemented by major players in the VR industry. These results underscore the severity of the vulnerability and its potential impact on millions of VR social platform users.

Remote Keylogging Attacks in Multi-user VR Applications

TL;DR

This study proposes a remote attack that utilizes the avatar rendering information collected from an adversary's game clients to extract user-typed secrets like credit card information, passwords, or private conversations from multi-user VR applications.

Abstract

As Virtual Reality (VR) applications grow in popularity, they have bridged distances and brought users closer together. However, with this growth, there have been increasing concerns about security and privacy, especially related to the motion data used to create immersive experiences. In this study, we highlight a significant security threat in multi-user VR applications, which are applications that allow multiple users to interact with each other in the same virtual space. Specifically, we propose a remote attack that utilizes the avatar rendering information collected from an adversary's game clients to extract user-typed secrets like credit card information, passwords, or private conversations. We do this by (1) extracting motion data from network packets, and (2) mapping motion data to keystroke entries. We conducted a user study to verify the attack's effectiveness, in which our attack successfully inferred 97.62% of the keystrokes. Besides, we performed an additional experiment to underline that our attack is practical, confirming its effectiveness even when (1) there are multiple users in a room, and (2) the attacker cannot see the victims. Moreover, we replicated our proposed attack on four applications to demonstrate the generalizability of the attack. Lastly, we proposed a defense against the attack, which has been implemented by major players in the VR industry. These results underscore the severity of the vulnerability and its potential impact on millions of VR social platform users.
Paper Structure (38 sections, 20 figures, 7 tables)

This paper contains 38 sections, 20 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Threat Model
  4. Approach
  5. Step 1: Packet Extraction
  6. Step 2: Field Extraction
  7. Step 3: Semantics Extraction
  8. Step 4: Click Position Extraction
  9. Reverse Engineering Challenges
  10. Precise Input Control
  11. Precise Measurement
  12. Data Collection
  13. Evaluation
  14. Our Attack is Robust Under Traffic Loss
  15. Keyboard Layout Affects Attack Accuracy
  16. ...and 23 more sections

Figures (20)

  • Figure 1: Example of user head motion in 6DOF sixdof.
  • Figure 2: Motion data flow in multi-user VR applications.
  • Figure 3: Example of a participant typing in Rec Room using a virtual keyboard.
  • Figure 4: Our approach consists of four extraction steps to extract keystrokes from network traffic.
  • Figure 5: By visualizing the extension of the cursor with blue arrow lines, we calculate its intersection with the keyboard, as indicated by the blue dots on the keys, to log the key selections.
  • ...and 15 more figures