CLAd-VR: Cognitive Load-based Adaptive Training for Machining Tasks in Virtual Reality

TL;DR

CLAd-VR addresses the lack of adaptive, cognitive-load-aware VR training in manufacturing by integrating wearable EEG-based sensing with an LSTM classifier to continuously estimate cognitive load and adapt instruction in real time. The system combines a Unity-based VR drilling module, synchronized EEG/behavior data streams, and a calibration-informed adaptation engine to deliver multimodal scaffolding that responds to high or low cognitive demand within $<100\,\text{ms}$ latency. Key contributions include the end-to-end architecture, the n-back calibration protocol establishing load thresholds $T_{\text{low}}$ and $T_{\text{high}}$, and a suite of adaptive strategies (visual, auditory, and embodied ghost-hand guidance) to maintain an optimal learning zone. This work lays a practical foundation for scalable, upskilling-focused VR training in manufacturing with potential improvements in engagement, skill acquisition, and transfer.

Abstract

With the growing need to effectively support workforce upskilling in the manufacturing sector, virtual reality is gaining popularity as a scalable training solution. However, most current systems are designed as static, step-by-step tutorials and do not adapt to a learner's needs or cognitive load, which is a critical factor in learning and longterm retention. We address this limitation with CLAd-VR, an adaptive VR training system that integrates realtime EEG-based sensing to measure the learner's cognitive load and adapt instruction accordingly, specifically for domain-specific tasks in manufacturing. The system features a VR training module for a precision drilling task, designed with multimodal instructional elements including animations, text, and video. Our cognitive load sensing pipeline uses a wearable EEG device to capture the trainee's neural activity, which is processed through an LSTM model to classify their cognitive load as low, optimal, or high in real time. Based on these classifications, the system dynamically adjusts task difficulty and delivers adaptive guidance using voice guidance, visual cues, or ghost hand animations. This paper introduces CLAd-VR system's architecture, including the EEG sensing hardware, real-time inference model, and adaptive VR interface.

Figures (2)

• Figure 1: Real-time EEG data stream across 14 channels with instances showing high CL (in red) and optimal CL (in blue).
• Figure 2: When the cognitive load is high, CLAd-VR adapts scaffolding to support the user using (a) Adaptation strategies based on the types of errors (b) example: arrow cues appear to guide the user’s actions; (c) example: ghost hands demonstrate the correct task steps to facilitate learning.