Toward Multimodal Privacy in XR: Design and Evaluation of Composite Privatization Methods for Gaze and Body Tracking Data

TL;DR

This paper tackles the privacy risks inherent in XR by evaluating real-time multimodal privacy mechanisms for gaze and body telemetry across casual and competitive applications. It systematically analyzes 14 mechanisms, proposes a context-aware evaluation framework, and demonstrates that carefully paired multimodal protections can reduce re-identification from the hundreds of percent range to roughly a quarter of prior leakage while preserving application usability. The authors validate findings on three datasets (OpenNEEDS, Chimera, Wilson et al.) and release the XR-Privacy SDK to facilitate practical adoption in Unity-based XR apps. The work underscores the need for modality-specific, context-aware privacy strategies and provides concrete guidance for deploying privacy mechanisms without compromising user experience.

Abstract

As extended reality (XR) systems become increasingly immersive and sensor-rich, they enable the collection of behavioral signals such as eye and body telemetry. These signals support personalized and responsive experiences and may also contain unique patterns that can be linked back to individuals. However, privacy mechanisms that naively pair unimodal mechanisms (e.g., independently apply privacy mechanisms for eye and body privatization) are often ineffective at preventing re-identification in practice. In this work, we systematically evaluate real-time privacy mechanisms for XR, both individually and in pair, across eye and body modalities. We assess privacy through re-identification rates and evaluate utility using numerical performance thresholds derived from existing literature to ensure real-time interaction requirements are met. We evaluated four eye and ten body mechanisms across multiple datasets, comprising up to 407 participants. Our results show that when carefully paired, multimodal mechanisms reduce re-identification rate from 80.3% to 26.3% in casual XR applications (e.g., VRChat and Job Simulator) and from 84.8% to 26.1% in competitive XR applications (e.g., Beat Saber and Synth Riders), all while maintaining acceptable performance based on established thresholds. To facilitate adoption, we additionally release XR Privacy SDK, an open-source toolkit enabling developers to integrate the privacy mechanisms into XR applications for real-time use. These findings underscore the potential of modality-specific and context-aware privacy strategies for protecting behavioral data in XR environments.

Figures (5)

• Figure 1: System overview. Behavioral signals from gaze, head, and hand sensors are accessed by XR applications and transmitted via modality-specific API functions (e.g., SendGazeStream()) as network telemetry. These streams may be received by external servers or users, where adversaries can perform re-identification attacks.
• Figure 2: Privacy-utility trade-off comparison for eye (left) and body (right) privacy mechanisms in Causal XR applications. Each plot presents the impact of different privacy methods on re-identification performance, measured by Rank-1 identification rate (IR) on the x-axis, and utility degradation, measured by angular distance (left) or Euclidean distance (right) on the y-axis. Points to the left indicate stronger privacy (lower re-identification rates), points toward the bottom indicate better utility preservation (minimal distance from original), and the bottom-left region represents optimal privacy-utility balance. Solid lines represent standalone mechanisms, while dashed lines represent DMM-based composite mechanisms. The original data points are indicated by circles. Dashed black lines represent literature thresholds shown to preserve interaction fidelity schuetz2020psychophysicswentzel2020improving.
• Figure 3: Comparison of privacy-utility trade-offs for body telemetry mechanisms. The plot illustrates how different privacy methods affect Rank-1 identification rate (IR) and the corresponding utility, measured by BeatSaber Score. Points to the left indicate stronger privacy (lower Rank-1 IR), points toward the top indicate better utility preservation (maximum preservation of BeatSaber score), and the bottom-left region represents optimal privacy-utility balance. Solid lines represent standalone mechanisms, while dash lines represent DMM-based composite mechanisms. The original data points are indicated by circles. Dashed black line represents literature thresholds shown to preserve interaction fidelity nair2023deep.
• Figure 4: GUI of XR Privacy. (A) Dropdown to select application type. (B) Slider to configure privacy level based on the tradeoff between privacy-utility. (C) Button to confirm selected settings.
• Figure 5: Mixed reality photo of a player using XR-Privacy SDK within a custom Unity-based virtual environment.