Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

I Know What You Did Last Summer: Identifying VR User Activity Through VR Network Traffic

Sheikh Samit Muhaimin, Spyridon Mastorakis

TL;DR

This work demonstrates that encrypted VR headset traffic contains distinguishable patterns that reveal both which VR application is in use and which user actions are being performed. Using data from 25 Meta Quest Pro apps and a straightforward ML pipeline (RF, DT, and MLP), the authors achieve about 92% app-identification accuracy and 91% activity-identification accuracy, with useful generalization even from traces of unknown apps. A key insight is that relatively small datasets (under 10 minutes per app) suffice to reach high identification performance, highlighting significant privacy risks without requiring payload access. The study combines feature engineering on traffic metrics with a robust evaluation framework, and discusses practical system trade-offs such as training/inference times and resource usage. Overall, the findings underscore the need for privacy-preserving VR architectures and informed policy to mitigate traffic-based user profiling and behavioral tracking in VR environments.

Abstract

Virtual Reality (VR) technology has gained substantial traction and has the potential to transform a number of industries, including education, entertainment, and professional sectors. Nevertheless, concerns have arisen about the security and privacy implications of VR applications and the impact that they might have on users. In this paper, we investigate the following overarching research question: can VR applications and VR user activities in the context of such applications (e.g., manipulating virtual objects, walking, talking, flying) be identified based on the (potentially encrypted) network traffic that is generated by VR headsets during the operation of VR applications? To answer this question, we collect network traffic data from 25 VR applications running on the Meta Quest Pro headset and identify characteristics of the generated network traffic, which we subsequently use to train off-the-shelf Machine Learning (ML) models. Our results indicate that through the use of ML models, we can identify the VR applications being used with an accuracy of 92.4% and the VR user activities performed with an accuracy of 91%. Furthermore, our results demonstrate that an attacker does not need to collect large amounts of network traffic data for each VR application to carry out such an attack. Specifically, an attacker only needs to collect less than 10 minutes of network traffic data for each VR application in order to identify applications with an accuracy higher than 90% and VR user activities with an accuracy higher than 88%.

I Know What You Did Last Summer: Identifying VR User Activity Through VR Network Traffic

TL;DR

This work demonstrates that encrypted VR headset traffic contains distinguishable patterns that reveal both which VR application is in use and which user actions are being performed. Using data from 25 Meta Quest Pro apps and a straightforward ML pipeline (RF, DT, and MLP), the authors achieve about 92% app-identification accuracy and 91% activity-identification accuracy, with useful generalization even from traces of unknown apps. A key insight is that relatively small datasets (under 10 minutes per app) suffice to reach high identification performance, highlighting significant privacy risks without requiring payload access. The study combines feature engineering on traffic metrics with a robust evaluation framework, and discusses practical system trade-offs such as training/inference times and resource usage. Overall, the findings underscore the need for privacy-preserving VR architectures and informed policy to mitigate traffic-based user profiling and behavioral tracking in VR environments.

Abstract

Virtual Reality (VR) technology has gained substantial traction and has the potential to transform a number of industries, including education, entertainment, and professional sectors. Nevertheless, concerns have arisen about the security and privacy implications of VR applications and the impact that they might have on users. In this paper, we investigate the following overarching research question: can VR applications and VR user activities in the context of such applications (e.g., manipulating virtual objects, walking, talking, flying) be identified based on the (potentially encrypted) network traffic that is generated by VR headsets during the operation of VR applications? To answer this question, we collect network traffic data from 25 VR applications running on the Meta Quest Pro headset and identify characteristics of the generated network traffic, which we subsequently use to train off-the-shelf Machine Learning (ML) models. Our results indicate that through the use of ML models, we can identify the VR applications being used with an accuracy of 92.4% and the VR user activities performed with an accuracy of 91%. Furthermore, our results demonstrate that an attacker does not need to collect large amounts of network traffic data for each VR application to carry out such an attack. Specifically, an attacker only needs to collect less than 10 minutes of network traffic data for each VR application in order to identify applications with an accuracy higher than 90% and VR user activities with an accuracy higher than 88%.
Paper Structure (36 sections, 6 figures, 12 tables)

This paper contains 36 sections, 6 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Problem and Motivation
  3. Threat Model
  4. Setting Up the Attack
  5. Relevant Data Collection
  6. Attacking Model(s)
  7. Re-Training (Fine Tuning) Models
  8. The Problem
  9. User Profiling
  10. Behavioral Tracking
  11. Social Engineering Attacks
  12. Data Breaches
  13. Legal and Ethical Implications
  14. Research Questions
  15. Methodology and Results
  16. ...and 21 more sections

Figures (6)

  • Figure 1: Overview of methodology.
  • Figure 2: Number of applications by genre
  • Figure 3: Confusion matrix for the identification of VR applications through a random forest classifier.
  • Figure 4: Confusion matrix for various user activities regardless of applications in the case of the random forest classifier.
  • Figure 5: An example of identification of VR user activities in a traffic trace containing unknown traffic data.
  • ...and 1 more figures