Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improvement in Facial Emotion Recognition using Synthetic Data Generated by Diffusion Model

Arnab Kumar Roy, Hemant Kumar Kathania, Adhitiya Sharma

TL;DR

This paper tackles the issue of data imbalance by incorporating synthetic data augmentation and leveraging the ResEmoteNet model to enhance the overall performance on facial emotion recognition task and underscores the potential of advanced generative models in FER research and applications.

Abstract

Facial Emotion Recognition (FER) plays a crucial role in computer vision, with significant applications in human-computer interaction, affective computing, and areas such as mental health monitoring and personalized learning environments. However, a major challenge in FER task is the class imbalance commonly found in available datasets, which can hinder both model performance and generalization. In this paper, we tackle the issue of data imbalance by incorporating synthetic data augmentation and leveraging the ResEmoteNet model to enhance the overall performance on facial emotion recognition task. We employed Stable Diffusion 2 and Stable Diffusion 3 Medium models to generate synthetic facial emotion data, augmenting the training sets of the FER2013 and RAF-DB benchmark datasets. Training ResEmoteNet with these augmented datasets resulted in substantial performance improvements, achieving accuracies of 96.47% on FER2013 and 99.23% on RAF-DB. These findings shows an absolute improvement of 16.68% in FER2013, 4.47% in RAF-DB and highlight the efficacy of synthetic data augmentation in strengthening FER models and underscore the potential of advanced generative models in FER research and applications. The source code for ResEmoteNet is available at https://github.com/ArnabKumarRoy02/ResEmoteNet

Improvement in Facial Emotion Recognition using Synthetic Data Generated by Diffusion Model

TL;DR

This paper tackles the issue of data imbalance by incorporating synthetic data augmentation and leveraging the ResEmoteNet model to enhance the overall performance on facial emotion recognition task and underscores the potential of advanced generative models in FER research and applications.

Abstract

Facial Emotion Recognition (FER) plays a crucial role in computer vision, with significant applications in human-computer interaction, affective computing, and areas such as mental health monitoring and personalized learning environments. However, a major challenge in FER task is the class imbalance commonly found in available datasets, which can hinder both model performance and generalization. In this paper, we tackle the issue of data imbalance by incorporating synthetic data augmentation and leveraging the ResEmoteNet model to enhance the overall performance on facial emotion recognition task. We employed Stable Diffusion 2 and Stable Diffusion 3 Medium models to generate synthetic facial emotion data, augmenting the training sets of the FER2013 and RAF-DB benchmark datasets. Training ResEmoteNet with these augmented datasets resulted in substantial performance improvements, achieving accuracies of 96.47% on FER2013 and 99.23% on RAF-DB. These findings shows an absolute improvement of 16.68% in FER2013, 4.47% in RAF-DB and highlight the efficacy of synthetic data augmentation in strengthening FER models and underscore the potential of advanced generative models in FER research and applications. The source code for ResEmoteNet is available at https://github.com/ArnabKumarRoy02/ResEmoteNet

Paper Structure

This paper contains 9 sections, 3 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. ResEmoteNet Architecture
  3. Loss Minimization
  4. Residual Feature Extraction
  5. Diffusion Based Data Augmentation
  6. Dataset and Experimental Setup
  7. Results and Discussion
  8. Comparison with previous study
  9. Conclusion

Figures (4)

  • Figure 1: Overview of ResEmoteNet for efficient facial emotion recognition.
  • Figure 2: Overall pipeline for data augmentation.
  • Figure 3: Examples of synthetic images representing each emotion class in the Facial Emotion Recognition (FER) task, generated using Diffusion Models based on the prompts listed in Table \ref{['tab:prompt-table']}.
  • Figure 4: Confusion matrices for the performance of the model on (a) Original FER2013, (b) FER2013 with Augmentation 4 (15000 samples in each class), (c) Original RAF-DB and (d) RAF-DB with Augmentation 4 (15000 samples in each class).