A Real-Time Error Prevention System for Gaze-Based Interaction in Virtual Reality Based on Anomaly Detection

TL;DR

The paper tackles the Midas touch problem in gaze-based VR by deploying a real-time EPS built on a TCNAE to detect anomalies in angular gaze velocity around selection events. By training on correct selections and using a reconstruction error threshold $th$, the system identifies erroneous inputs and prevents them without interrupting flow, evaluated in a VR visual-search game with Dwell Time, Gaze and Head, and Nod methods. Results show substantial reductions in incorrect selections and improved points for Dwell Time and Gaze and Head, with more modest benefits for Nod, and participants generally ratings EPS as helpful. The work demonstrates the practical viability of anomaly-based, real-time error prevention for gaze interfaces with implications for VR, AR, and assistive technologies, while underscoring the need for adaptive personalization to account for individual variability. The approach offers a scalable pathway to quieter, more trustworthy gaze interaction across immersive platforms.

Abstract

Gaze-based interaction enables intuitive, hands-free control in immersive environments, but remains susceptible to unintended inputs. We present a real-time error prevention system (EPS) that uses a temporal convolutional network autoencoder (TCNAE) to detect anomalies in gaze dynamics during selection tasks. In a visual search task in VR, 41 participants used three gaze-based methods - dwell time, gaze and head direction alignment, and nod - with and without EPS. The system reduced erroneous selections by up to 95% for dwell time and gaze and head, and was positively received by most users. Performance varied for nodding and between individuals, suggesting the need for adaptive systems. Objective metrics and subjective evaluations show that anomaly-based error prevention can improve gaze interfaces without disrupting interaction. These findings demonstrate the potential of anomaly-based error prevention for gaze interfaces and suggest applications in VR, AR, and assistive technologies.

Figures (6)

• Figure 1: Architecture of the tcnae. The network comprises an encoder (left), bottleneck (center), and decoder (right). The encoder and decoder each consist of stacked TCN blocks, where each block includes a 1D convolution, batch normalization, and relu activation (see TCNBlock, right). The number of input and output channels, kernel size, and dilation factor are indicated within each block. The bottleneck compresses the input sequence into a fixed-length latent vector, which is then reconstructed to the original shape. All convolutional layers preserve sequence length via appropriate padding.
• Figure 2: Velocity profiles of network input for the various interaction methods, represented as the median value of all participants at each point in time, with the shaded areas indicating the 25th and 75th percentiles.
• Figure 3: Illustration of ve: (a) high score list, (b) current points and lap counter (top right), (c) timer display, and (d) robots with targets, with the main target marked by the “C” at the bottom.
• Figure 4: Boxplots of participants' responses to the questionnaire for each method. Each subplot corresponds to a questionnaire item, with the methods shown on the y-axis and the response scale on the x-axis. The figure illustrates the distribution, central tendency, and variability of all participants' ratings.
• Figure 5: Comparison of task performance with the eps inactive and active. Subfigure (a) shows the number of incorrect selections, while subfigure (b) displays the median achieved points.