Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Penetration Vision through Virtual Reality Headsets: Identifying 360-degree Videos from Head Movements

Anh Nguyen, Xiaokuan Zhang, Zhisheng Yan

TL;DR

This work reveals a new privacy vulnerability in VR: 360° videos viewed through HMDs leak their titles through users’ head movements observed by an external camera. It introduces Intrude, a two-component attack consisting of an HMD-based head movement estimation (HME) module and a video saliency-based trace-fingerprint matching (TFM) framework that links head-trace data to saliency fingerprints of candidate videos. Empirical results show top-1 identification accuracy of 96% (and 99% top-2/top-3) in indoor settings, with robustness across recording conditions and open-world scenarios, highlighting substantial privacy risk in public VR usage. The paper also discusses limitations and mitigations, including the need for defenses that consider VR gesture dynamics and roll-axis initialization to reduce leakage while preserving user experience.

Abstract

In this paper, we present the first contactless side-channel attack for identifying 360 videos being viewed in a Virtual Reality (VR) Head Mounted Display (HMD). Although the video content is displayed inside the HMD without any external exposure, we observe that user head movements are driven by the video content, which creates a unique side channel that does not exist in traditional 2D videos. By recording the user whose vision is blocked by the HMD via a malicious camera, an attacker can analyze the correlation between the user's head movements and the victim video to infer the video title. To exploit this new vulnerability, we present INTRUDE, a system for identifying 360 videos from recordings of user head movements. INTRUDE is empowered by an HMD-based head movement estimation scheme to extract a head movement trace from the recording and a video saliency-based trace-fingerprint matching framework to infer the video title. Evaluation results show that INTRUDE achieves over 96% of accuracy for video identification and is robust under different recording environments. Moreover, INTRUDE maintains its effectiveness in the open-world identification scenario.

Penetration Vision through Virtual Reality Headsets: Identifying 360-degree Videos from Head Movements

TL;DR

This work reveals a new privacy vulnerability in VR: 360° videos viewed through HMDs leak their titles through users’ head movements observed by an external camera. It introduces Intrude, a two-component attack consisting of an HMD-based head movement estimation (HME) module and a video saliency-based trace-fingerprint matching (TFM) framework that links head-trace data to saliency fingerprints of candidate videos. Empirical results show top-1 identification accuracy of 96% (and 99% top-2/top-3) in indoor settings, with robustness across recording conditions and open-world scenarios, highlighting substantial privacy risk in public VR usage. The paper also discusses limitations and mitigations, including the need for defenses that consider VR gesture dynamics and roll-axis initialization to reduce leakage while preserving user experience.

Abstract

In this paper, we present the first contactless side-channel attack for identifying 360 videos being viewed in a Virtual Reality (VR) Head Mounted Display (HMD). Although the video content is displayed inside the HMD without any external exposure, we observe that user head movements are driven by the video content, which creates a unique side channel that does not exist in traditional 2D videos. By recording the user whose vision is blocked by the HMD via a malicious camera, an attacker can analyze the correlation between the user's head movements and the victim video to infer the video title. To exploit this new vulnerability, we present INTRUDE, a system for identifying 360 videos from recordings of user head movements. INTRUDE is empowered by an HMD-based head movement estimation scheme to extract a head movement trace from the recording and a video saliency-based trace-fingerprint matching framework to infer the video title. Evaluation results show that INTRUDE achieves over 96% of accuracy for video identification and is robust under different recording environments. Moreover, INTRUDE maintains its effectiveness in the open-world identification scenario.
Paper Structure (40 sections, 7 equations, 18 figures)

This paper contains 40 sections, 7 equations, 18 figures.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. 360° Videos and Head Movement
  4. Video Fingerprint and Saliency Maps
  5. Overview of Intrude
  6. Threat Model
  7. Challenges and Key Components
  8. HMD-based Head Movement Estimation (HME)
  9. Identifying the Start of Video Playback
  10. Estimating Head Orientation
  11. Converting Head Orientation Vectors
  12. Video Saliency-based Trace-Fingerprint Matching (TFM)
  13. Creating Fingerprints for 360° Videos
  14. Matching Victim Trace with Fingerprints
  15. Making Video Identification Decisions
  16. ...and 25 more sections

Figures (18)

  • Figure 1: The head movement traces of different users (right figure, 4 colors) on the same video may be different in the time domain, but they do present a similar head movement pattern on regions of interest (right figure, brighter areas).
  • Figure 2: The saliency maps (bottom) uniquely identifies 360° videos (top) through regions of interest.
  • Figure 3: Sample attack scenarios -- recording with a smartphone rear camera (left) and front camera (right).
  • Figure 4: Overview of the proposed Intrude system.
  • Figure 5: The head orientation estimation model.
  • ...and 13 more figures