Learner Attentiveness and Engagement Analysis in Online Education Using Computer Vision
Sharva Gogawale, Madhura Deshpande, Parteek Kumar, Irad Ben-Gal
TL;DR
This work tackles real-time estimation of learner attentiveness in online education using computer vision by framing a multiclass, multioutput classification of four affective states on the DAiSEE dataset and deriving an Attentiveness Index. It introduces two parallel architectures (CNN-based and EfficientNet-based) with four parallel branches, trained with categorical focal loss to handle data imbalance, and aggregates the outputs into an Attentiveness Index via $A_i = -0.598 \cdot B + 1.539 \cdot E + 0.334 \cdot C - 0.085 \cdot F$. An end-to-end online platform delivers real-time analytics to instructors, including class-wide alerts and detailed visualizations. Experimental results show superior state-wise accuracy compared to prior methods, demonstrating a cost-effective, deployable system for online learning environments.
Abstract
In recent times, online education and the usage of video-conferencing platforms have experienced massive growth. Due to the limited scope of a virtual classroom, it may become difficult for instructors to analyze learners' attention and comprehension in real time while teaching. In the digital mode of education, it would be beneficial for instructors to have an automated feedback mechanism to be informed regarding learners' attentiveness at any given time. This research presents a novel computer vision-based approach to analyze and quantify learners' attentiveness, engagement, and other affective states within online learning scenarios. This work presents the development of a multiclass multioutput classification method using convolutional neural networks on a publicly available dataset - DAiSEE. A machine learning-based algorithm is developed on top of the classification model that outputs a comprehensive attentiveness index of the learners. Furthermore, an end-to-end pipeline is proposed through which learners' live video feed is processed, providing detailed attentiveness analytics of the learners to the instructors. By comparing the experimental outcomes of the proposed method against those of previous methods, it is demonstrated that the proposed method exhibits better attentiveness detection than state-of-the-art methods. The proposed system is a comprehensive, practical, and real-time solution that is deployable and easy to use. The experimental results also demonstrate the system's efficiency in gauging learners' attentiveness.