Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Deep Learning Model to Estimate Cognitive Effort from fNIRS Signals

Shayla Sharmin, Roghayeh Leila Barmaki

TL;DR

This paper addresses estimating cognitive effort during learning by combining fNIRS-derived brain signals with performance predictions. It introduces a hybrid CNN-GRU DeepNet to predict quiz performance from fNIRS signals collected in an educational game, and uses the predicted scores to compute cognitive effort measures such as Relative Neural Efficiency (RNE) and Relative Neural Involvement (RNI). The approach achieves 73.08% accuracy (F1 0.82) in predicting performance and demonstrates that predicted RNE and RNI track actual trends with high correlation, despite moderate overall accuracy. These results support the feasibility of neural-feedback–driven adaptive learning systems and point to real-time, personalized educational applications.

Abstract

This study estimates cognitive effort based on functional near-infrared spectroscopy data and performance scores using a hybrid DeepNet model. The estimation of cognitive effort enables educators to modify material to enhance learning effectiveness and student engagement. In this study, we collected oxygenated hemoglobin using functional near-infrared spectroscopy during an educational quiz game. Participants (n=16) responded to 16 questions in a Unity-based educational game, each within a 30-second response time limit. We used DeepNet models to predict the performance score from the oxygenated hemoglobin, and compared traditional machine learning and DeepNet models to determine which approach provides better accuracy in predicting performance scores. The result shows that the proposed CNN-GRU gives better performance with 73% than other models. After the prediction, we used the predicted score and the oxygenated hemoglobin to observe cognitive effort by calculating relative neural efficiency and involvement in our test cases. Our result shows that even with moderate accuracy, the predicted cognitive effort closely follow the actual trends. This findings can be helpful in designing and improving learning environments and provide valuable insights into learning materials.

Hybrid Deep Learning Model to Estimate Cognitive Effort from fNIRS Signals

TL;DR

This paper addresses estimating cognitive effort during learning by combining fNIRS-derived brain signals with performance predictions. It introduces a hybrid CNN-GRU DeepNet to predict quiz performance from fNIRS signals collected in an educational game, and uses the predicted scores to compute cognitive effort measures such as Relative Neural Efficiency (RNE) and Relative Neural Involvement (RNI). The approach achieves 73.08% accuracy (F1 0.82) in predicting performance and demonstrates that predicted RNE and RNI track actual trends with high correlation, despite moderate overall accuracy. These results support the feasibility of neural-feedback–driven adaptive learning systems and point to real-time, personalized educational applications.

Abstract

This study estimates cognitive effort based on functional near-infrared spectroscopy data and performance scores using a hybrid DeepNet model. The estimation of cognitive effort enables educators to modify material to enhance learning effectiveness and student engagement. In this study, we collected oxygenated hemoglobin using functional near-infrared spectroscopy during an educational quiz game. Participants (n=16) responded to 16 questions in a Unity-based educational game, each within a 30-second response time limit. We used DeepNet models to predict the performance score from the oxygenated hemoglobin, and compared traditional machine learning and DeepNet models to determine which approach provides better accuracy in predicting performance scores. The result shows that the proposed CNN-GRU gives better performance with 73% than other models. After the prediction, we used the predicted score and the oxygenated hemoglobin to observe cognitive effort by calculating relative neural efficiency and involvement in our test cases. Our result shows that even with moderate accuracy, the predicted cognitive effort closely follow the actual trends. This findings can be helpful in designing and improving learning environments and provide valuable insights into learning materials.

Paper Structure

This paper contains 14 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Experiment
  4. User Study
  5. Cognitive Effort Measure
  6. Data Acquisition
  7. Data Pre-processing
  8. Model Architecture
  9. Results
  10. Prediction of Performance Score
  11. Feature Attribution Analysis (Interpretability)
  12. Estimation of Cognitive Effort
  13. Discussion
  14. Conclusion

Figures (4)

  • Figure 1: DeepNet Model Architecture. We fed fNIRS data into our model and to get performance score as output
  • Figure 2: Correlation between GRU features and PCA components
  • Figure 3: SHAP summary plot showing GRU feature importance in prediction
  • Figure 4: Scatter plots comparing actual vs. predicted values of RNE (left) and RNI (right). The dotted diagonal line indicates perfect prediction. Data points cluster tightly around the line, demonstrating strong predictive accuracy.