Emotion Recognition Using Convolutional Neural Networks
Shaoyuan Xu, Yang Cheng, Qian Lin, Jan P. Allebach
TL;DR
This paper addresses automatic emotion recognition from facial expressions, seeking both accurate category classification and intensity estimation in real-time. It introduces a compact CNN architecture derived from VGG-S, augmented by targeted face alignment and data augmentation, and trains on a combination of public datasets (CK+, Jaffe, MUG-FED, KDEF) plus a self-collected Emotion Intensity In the Wild set. The authors implement both a classification model and a regression model to output emotion labels and their intensities, reporting 85% validation accuracy for classification and RMSE around 0.13 for intensity regression, with a real-time demo achieving frame-level inference on standard hardware. The work demonstrates practical viability of deep-learning-based emotion recognition in-the-wild and in real-time, and contributes a labeled in-the-wild dataset for intensity annotation.
Abstract
Emotion has an important role in daily life, as it helps people better communicate with and understand each other more efficiently. Facial expressions can be classified into 7 categories: angry, disgust, fear, happy, neutral, sad and surprise. How to detect and recognize these seven emotions has become a popular topic in the past decade. In this paper, we develop an emotion recognition system that can apply emotion recognition on both still images and real-time videos by using deep learning. We build our own emotion recognition classification and regression system from scratch, which includes dataset collection, data preprocessing , model training and testing. Given a certain image or a real-time video, our system is able to show the classification and regression results for all of the 7 emotions. The proposed system is tested on 2 different datasets, and achieved an accuracy of over 80\%. Moreover, the result obtained from real-time testing proves the feasibility of implementing convolutional neural networks in real time to detect emotions accurately and efficiently.