Real Time Emotion Analysis Using Deep Learning for Education, Entertainment, and Beyond

TL;DR

The paper addresses real-time facial expression recognition for emoji translation across education, entertainment, and related domains. It combines OpenCV-based face detection with CNN and ResNet architectures (including CNN, ResNet18, and LSTM variants), evaluated on FER2013 and FER2013+ with data augmentation. A GUI prototype using tkinter integrates static image and live webcam predictions, leveraging Haar cascades, OpenCV, and TensorFlow. A student engagement monitoring system is proposed to compute per-student engagement scores from facial cues, and the work discusses broader applications and avenues for improving robustness and real-time performance.

Abstract

The significance of emotion detection is increasing in education, entertainment, and various other domains. We are developing a system that can identify and transform facial expressions into emojis to provide immediate feedback.The project consists of two components. Initially, we will employ sophisticated image processing techniques and neural networks to construct a deep learning model capable of precisely categorising facial expressions. Next, we will develop a basic application that records live video using the camera on your device. The app will utilise a sophisticated model to promptly analyse facial expressions and promptly exhibit corresponding emojis.Our objective is to develop a dynamic tool that integrates deep learning and real-time video processing for the purposes of online education, virtual events, gaming, and enhancing user experience. This tool enhances interactions and introduces novel emotional intelligence technologies.

Figures (18)

• Figure 1: Data Imbalance
• Figure 2: Accuracy Using Different Approaches.
• Figure 3: Accuracy v/s Epochs for Training and Validation sets
• Figure 4: Loss v/s Epochs for Training and Validation sets
• Figure 5: ResNet18 Initial architecture