Non-intrusive and Unconstrained Keystroke Inference in VR Platforms via Infrared Side Channel

TL;DR

This work reveals a novel infrared side channel in VR headset controller constellation tracking that enables non-intrusive reconstruction of virtual keystrokes. It presents VRecKey, a model-free attack using a custom 2D IR sensor array to capture controller IR emissions, calibrate keyboard coordinates, generate heatmaps, and infer both character-level and unconstrained word-level keystrokes, aided by an LLM for correction. Extensive evaluations on Meta Oculus Quest 2 and PICO 4 All-in-One show high accuracy (character-level T-1 85.8%, T-3 94.2%; word-level T-1 81.7%, T-3 90.5%) across real-world scenarios and practical factors (distance, orientation, speed, and single/multiple IR sources). The results underscore a significant privacy risk in VR platforms and motivate countermeasures such as encrypted IR transmission and shuffled keyboards, highlighting the need for defense research as VR devices become more widespread. Overall, the paper demonstrates a scalable, non-intrusive attack that broadens the threat landscape for keystroke inference in VR and calls for mitigations to protect user credentials and sensitive inputs.

Abstract

Virtual Reality (VR) technologies are increasingly employed in numerous applications across various areas. Therefore, it is essential to ensure the security of interactions between users and VR devices. In this paper, we disclose a new side-channel leakage in the constellation tracking system of mainstream VR platforms, where the infrared (IR) signals emitted from the VR controllers for controller-headset interactions can be maliciously exploited to reconstruct unconstrained input keystrokes on the virtual keyboard non-intrusively. We propose a novel keystroke inference attack named VRecKey to demonstrate the feasibility and practicality of this novel infrared side channel. Specifically, VRecKey leverages a customized 2D IR sensor array to intercept ambient IR signals emitted from VR controllers and subsequently infers (i) character-level key presses on the virtual keyboard and (ii) word-level keystrokes along with their typing trajectories. We extensively evaluate the effectiveness of VRecKey with two commercial VR devices, and the results indicate that it can achieve over 94.2% and 90.5% top-3 accuracy in inferring character-level and word-level keystrokes with varying lengths, respectively. In addition, empirical results show that VRecKey is resilient to several practical impact factors and presents effectiveness in various real-world scenarios, which provides a complementary and orthogonal attack surface for the exploration of keystroke inference attacks in VR platforms.

Figures (26)

• Figure 1: Illustration of VRecKey attack: The victim types the virtual keyboard to input the keystroke "HELLO". Meanwhile, the attacker leverages a 2D IR sensor array to capture the IR signals emitted to the ambient environment and reconstruct the heatmaps to infer the virtual keystroke and its trajectory.
• Figure 2: Illustration of infrared LEDs embedded in commodity VR controllers (denoted as blue dots on the ring) and the constellation tracking system in VR devices (\ref{['subsec:infrared_led_vr']}).
• Figure 3: Preliminary of IR sensors, including the circuit to capture IR signals and the recorded voltage signal (\ref{['subsec:principle_infrared_sensors']}).
• Figure 4: Overview of VRecKey (\ref{['subsec:attack_overview']}).
• Figure 5: Three real-world attack scenarios (\ref{['sec:threat_model']}), including a concealed attack with one-way film, a reflection-based attack, and an attack in a low-visibility environment.