A Framework to Prevent Biometric Data Leakage in the Immersive Technologies Domain

TL;DR

The paper addresses biometric data leakage in immersive technologies by proposing a two-stage privacy framework consisting of Stage 1 feature extraction and classification to label biometric versus non-biometric signals, followed by Stage 2 filtering to block biometric data. It employs classifiers such as Random Forest, XGBoost, and LightGBM, with a feature set including entropy and spectral measures, and validates the approach across six datasets (ECG, EEG, body movement, audio) and the MultiModalBioAudio collection. The evaluation demonstrates fast training/testing and robustness to distortions, with SHAP-based analysis highlighting the mean power spectral density as a key discriminator; results indicate high classification accuracy (approaching 99% under certain conditions) and effective leakage prevention without relying on metadata. This framework offers a practical, privacy-preserving pathway to reconcile biometric data collection with user rights and regulatory expectations in VR/AR ecosystems, and lays groundwork for firmware integration and policy-aligned future work.

Abstract

Doubtlessly, the immersive technologies have potential to ease people's life and uplift economy, however the obvious data privacy risks cannot be ignored. For example, a participant wears a 3D headset device which detects participant's head motion to track the pose of participant's head to match the orientation of camera with participant's eyes positions in the real-world. In a preliminary study, researchers have proved that the voice command features on such headsets could lead to major privacy leakages. By analyzing the facial dynamics captured with the motion sensors, the headsets suffer security vulnerabilities revealing a user's sensitive speech without user's consent. The psychography data (such as voice command features, facial dynamics, etc.) is sensitive data and it should not be leaked out of the device without users consent else it is a privacy breach. To the best of our literature review, the work done in this particular research problem is very limited. Motivated from this, we develop a simple technical framework to mitigate sensitive data (or biometric data) privacy leaks in immersive technology domain. The performance evaluation is conducted in a robust way using six data sets, to show that the proposed solution is effective and feasible to prevent this issue.

Figures (6)

• Figure 1: The high level view of the proposed approach. The approach comprises two primary modules: a feature extractor and classifiers. The feature extractor processes raw signal data, extracting a range of features. Subsequently, these extracted features serve as input for the classifiers, which employ algorithms like random forest, XGBoost, and LightBGM. The classifiers classify the signals or data into its correct category, and the output is routed through a filter. The filter selectively allows the passage of non-biometric signals, screening out any biometric signals.
• Figure 2: Feature importance was compared across Random Forest, XGBoost, and LightGBM classifiers using the MultiModalBioAudio dataset, SHAP values were computed following the execution of a forward feature selection algorithm aimed at identifying the most influential features for each model. The analysis focused exclusively on the impact of features selected through the forward feature selection process.
• Figure 3: Accuracy of the Random Forest Classifier, XGBoost Classifier and LightGBM Classifier for different features and sample size.
• Figure 4: Accuracy and F1 score of the Random Forest Classifier, XGBoost Classifier, and LightGBM Classifier for different factors of a) horizontal scaling and b) vertical scaling. The signals were horizontally and vertically scaled corresponding (x-axis) to the equations $z(t) = s(\alpha· t)$ and $z(t) = \alpha · s(t)$ where z(t) is the scaled signal $\alpha$ is the scaling factor and s(t) is the original signal.
• Figure 5: Accuracy and F1 score of the Random Forest Classifier, XGBoost Classifier and LightGBM Classifier for different standard deviation values for AWGN. It is worth noting that the LightGBM classifier and XGBoost classifier exhibit similar behavior. Noise was added to the original signal corresponding to the equation $z(t) = s(t) +\mathcal{N} (x; 0, \sigma)$ where z(t) is the output signal, s(t) is the original signal and $\mathcal{N}$ represents the normal distribution.