Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Intelligent VR Training: A Physiological Adaptation Framework for Cognitive Load and Stress Detection

Mahsa Nasri

TL;DR

This work addresses detecting cognitive load and stress in VR training using eye-tracking and HRV to drive real-time adaptation. It proposes a three-phase methodology beginning with Stroop-based labeling to train ML models, followed by fine-tuning on new users and integrating privacy-aware detection. The study compares adaptive VR against privacy-preserving and non-adaptive baselines, evaluating learning performance, engagement, and workload with metrics such as NASA-TLX, presence, and task performance. The anticipated impact spans education, training, and healthcare by enabling scalable, privacy-conscious adaptive VR systems with real-time inference.

Abstract

Adaptive Virtual Reality (VR) systems have the potential to enhance training and learning experiences by dynamically responding to users' cognitive states. This research investigates how eye tracking and heart rate variability (HRV) can be used to detect cognitive load and stress in VR environments, enabling real-time adaptation. The study follows a three-phase approach: (1) conducting a user study with the Stroop task to label cognitive load data and train machine learning models to detect high cognitive load, (2) fine-tuning these models with new users and integrating them into an adaptive VR system that dynamically adjusts training difficulty based on physiological signals, and (3) developing a privacy-aware approach to detect high cognitive load and compare this with the adaptive VR in Phase two. This research contributes to affective computing and adaptive VR using physiological sensing, with applications in education, training, and healthcare. Future work will explore scalability, real-time inference optimization, and ethical considerations in physiological adaptive VR.

Towards Intelligent VR Training: A Physiological Adaptation Framework for Cognitive Load and Stress Detection

TL;DR

This work addresses detecting cognitive load and stress in VR training using eye-tracking and HRV to drive real-time adaptation. It proposes a three-phase methodology beginning with Stroop-based labeling to train ML models, followed by fine-tuning on new users and integrating privacy-aware detection. The study compares adaptive VR against privacy-preserving and non-adaptive baselines, evaluating learning performance, engagement, and workload with metrics such as NASA-TLX, presence, and task performance. The anticipated impact spans education, training, and healthcare by enabling scalable, privacy-conscious adaptive VR systems with real-time inference.

Abstract

Adaptive Virtual Reality (VR) systems have the potential to enhance training and learning experiences by dynamically responding to users' cognitive states. This research investigates how eye tracking and heart rate variability (HRV) can be used to detect cognitive load and stress in VR environments, enabling real-time adaptation. The study follows a three-phase approach: (1) conducting a user study with the Stroop task to label cognitive load data and train machine learning models to detect high cognitive load, (2) fine-tuning these models with new users and integrating them into an adaptive VR system that dynamically adjusts training difficulty based on physiological signals, and (3) developing a privacy-aware approach to detect high cognitive load and compare this with the adaptive VR in Phase two. This research contributes to affective computing and adaptive VR using physiological sensing, with applications in education, training, and healthcare. Future work will explore scalability, real-time inference optimization, and ethical considerations in physiological adaptive VR.

Paper Structure

This paper contains 14 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Cognitive Load, Emotion, and Physiological Responses in VR
  4. VR Adaptation Mechanisms
  5. Computational Models for Adaptive VR
  6. Privacy and Ethical Concerns in Physiological Adaptation
  7. Research Goals and Methodology
  8. Phase 1: Cognitive Load Detection Using Physiological Signals
  9. Phase 2: Machine Learning-Based Real-Time Adaptation
  10. Phase 3: Privacy-Aware Cognitive Load Detection
  11. Results to Date
  12. Next Steps and Long-Term Goals
  13. Milestones with Success Criteria and Dissemination Plan
  14. Supervisors and PhD Program

Figures (2)

  • Figure 1: Phase 1: Initial data collection pipeline where users undergo VR calibration, including baseline measurements and the Stroop task, followed by VR training to collect physiological signals for machine learning model training.
  • Figure 2: Phase 2: Adaptive VR system where trained models are fine-tuned, real-time task difficulty is adjusted based on cognitive load detection, and machine learning models are updated iteratively for improved adaptation.