Designing and Evaluating an Adaptive Virtual Reality System using EEG Frequencies to Balance Internal and External Attention States

TL;DR

This paper tackles VR cognitive workload and engagement by proposing an EEG-driven adaptive VR system that balances internal and external attention during a visual working memory task. It uses alpha and theta EEG bands to dynamically adjust peripheral distractors (NPCs) in real time, comparing two strategies that optimize either internal or external attention. The study demonstrates the feasibility of online adaptation, achieving an offline-to-online classification accuracy of about $0.86$ with an LDA model and showing improved WM accuracy under internal-attention optimization while external-attention optimization increases perceived workload. These results highlight the potential for physiologically driven, attention-aware VR to enhance task performance and user experience, with openly shared datasets and code to foster further development.

Abstract

Virtual reality finds various applications in productivity, entertainment, and training scenarios requiring working memory and attentional resources. Working memory relies on prioritizing relevant information and suppressing irrelevant information through internal attention, which is fundamental for successful task performance and training. Today, virtual reality systems do not account for the impact of working memory loads resulting in over or under-stimulation. In this work, we designed an adaptive system based on EEG correlates of external and internal attention to support working memory task performance. Here, participants engaged in a visual working memory N-Back task, and we adapted the visual complexity of distracting surrounding elements. Our study first demonstrated the feasibility of EEG frontal theta and parietal alpha frequency bands for dynamic visual complexity adjustments. Second, our adaptive system showed improved task performance and diminished perceived workload compared to a reverse adaptation. Our results show the effectiveness of the proposed adaptive system, allowing for the optimization of distracting elements in high-demanding conditions. Adaptive systems based on alpha and theta frequency bands allow for the regulation of attentional and executive resources to keep users engaged in a task without resulting in cognitive overload.

Figures (9)

• Figure 1: Adaptation Methodology for the two adaptive systems based on the increase and decrease of the alpha and theta frequency bands and their relevance to internal and external attentional states.
• Figure 2: Architecture of the two adaptive systems. The Stream of NPCs adapts based on alpha and theta variation in two different time windows ($w_1$ and $w_2$), each lasting 20s. If the change is bigger than the decision threshold of 15%, the NPC stream is either increased by +16 or decreased by -8 NPCs. The Positive Adaptation system (a) aims at optimizing internal attention, while the Negative Adaptation system (b) targets external attention.
• Figure 3: Game VR Capture of the experimental tasks. In the Visual Monitoring task (a), participants were exposed to a Stream of NPCs and asked to monitor, i.e., follow with their gaze NPCs with a specific colour. In the N-Back No Adaptation (b), participants actively interact with a sequence of spheres presented on a marble-like pillar and have to place them into either the left or right bucket. The placement of each sphere is determined by its color, and the sphere's color presented two steps prior (N=2). The sphere has to be placed on the left if the color is different and on the right bucket, if the color is the same.
• Figure 4: Experiment Procedure. The experiment encompassed six different blocks. In between blocks, participants filled in NASA-TLX and GEQ subscales and observed a three-minute pause in VR. Blocks order was randomized for the Visual Monitoring, N-Back with No Adaptation and N-Back with Positive or Negative Adaptation. In the first block, participants maintained their eyes closed to compute the Individual Alpha Frequency (IAF). In the Resting state block, participants relaxed in the neutral VR environment with distracting elements. After those two blocks, participants experienced the experimental tasks (Visual Monitoring, N-Back No Adaptation, N-Back Positive Adaptation, N-Back Negative Adaptation block) in a randomized order. Refer to \ref{['ssec:arch_ada']} for a complete description of the adaptive systems.
• Figure 5: Stream Visualization. Here, we depict the average evolution over time of the Stream for the two adaptive systems. The Positive Adaptation averaged on $133.17$ NPCs per minute while the Negative Adaptation on $161.48$ NPCs.