Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EmoGRACE: Aspect-based emotion analysis for social media data

Christina Zorenböhmer, Sebastian Schmidt, Bernd Resch

TL;DR

This work tackles the lack of labeled data for aspect-based emotion analysis on social media by constructing the first English ABEA training set (2,621 Tweets) with group annotation and majority voting. It repurposes and fine-tunes the GRACE ABSA model for joint ATE and AEC, achieving a plateau at an average F1 of 50.8% across configurations, with 70.1% for ATE alone and 46.9% for the joint extraction. The study identifies data scarcity and higher task complexity as the main bottlenecks, noting overfitting risks and limited generalization despite hyperparameter tuning. It suggests data augmentation and the use of generative foundation models as promising directions, and points to potential domain-specific applications such as disaster response or public-health monitoring to realize ABEE in practice.

Abstract

While sentiment analysis has advanced from sentence to aspect-level, i.e., the identification of concrete terms related to a sentiment, the equivalent field of Aspect-based Emotion Analysis (ABEA) is faced with dataset bottlenecks and the increased complexity of emotion classes in contrast to binary sentiments. This paper addresses these gaps, by generating a first ABEA training dataset, consisting of 2,621 English Tweets, and fine-tuning a BERT-based model for the ABEA sub-tasks of Aspect Term Extraction (ATE) and Aspect Emotion Classification (AEC). The dataset annotation process was based on the hierarchical emotion theory by Shaver et al. [1] and made use of group annotation and majority voting strategies to facilitate label consistency. The resulting dataset contained aspect-level emotion labels for Anger, Sadness, Happiness, Fear, and a None class. Using the new ABEA training dataset, the state-of-the-art ABSA model GRACE by Luo et al. [2] was fine-tuned for ABEA. The results reflected a performance plateau at an F1-score of 70.1% for ATE and 46.9% for joint ATE and AEC extraction. The limiting factors for model performance were broadly identified as the small training dataset size coupled with the increased task complexity, causing model overfitting and limited abilities to generalize well on new data.

EmoGRACE: Aspect-based emotion analysis for social media data

TL;DR

This work tackles the lack of labeled data for aspect-based emotion analysis on social media by constructing the first English ABEA training set (2,621 Tweets) with group annotation and majority voting. It repurposes and fine-tunes the GRACE ABSA model for joint ATE and AEC, achieving a plateau at an average F1 of 50.8% across configurations, with 70.1% for ATE alone and 46.9% for the joint extraction. The study identifies data scarcity and higher task complexity as the main bottlenecks, noting overfitting risks and limited generalization despite hyperparameter tuning. It suggests data augmentation and the use of generative foundation models as promising directions, and points to potential domain-specific applications such as disaster response or public-health monitoring to realize ABEE in practice.

Abstract

While sentiment analysis has advanced from sentence to aspect-level, i.e., the identification of concrete terms related to a sentiment, the equivalent field of Aspect-based Emotion Analysis (ABEA) is faced with dataset bottlenecks and the increased complexity of emotion classes in contrast to binary sentiments. This paper addresses these gaps, by generating a first ABEA training dataset, consisting of 2,621 English Tweets, and fine-tuning a BERT-based model for the ABEA sub-tasks of Aspect Term Extraction (ATE) and Aspect Emotion Classification (AEC). The dataset annotation process was based on the hierarchical emotion theory by Shaver et al. [1] and made use of group annotation and majority voting strategies to facilitate label consistency. The resulting dataset contained aspect-level emotion labels for Anger, Sadness, Happiness, Fear, and a None class. Using the new ABEA training dataset, the state-of-the-art ABSA model GRACE by Luo et al. [2] was fine-tuned for ABEA. The results reflected a performance plateau at an F1-score of 70.1% for ATE and 46.9% for joint ATE and AEC extraction. The limiting factors for model performance were broadly identified as the small training dataset size coupled with the increased task complexity, causing model overfitting and limited abilities to generalize well on new data.

Paper Structure

This paper contains 27 sections, 3 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Theories of Emotion
  4. Training datasets for ABSA & ABEA
  5. Models for ABSA
  6. Models for ABEA
  7. Methodology
  8. Data
  9. Training dataset annotation
  10. Emotion detection
  11. Aspect term detection
  12. Inter-annotator agreement
  13. Dataset statistics
  14. ABEA model fine-tuning
  15. GRACE: Gradient-Harmonized and Cascaded Labeling for Aspect-Based Sentiment Analysis
  16. ...and 12 more sections

Figures (8)

  • Figure 1: Two sub-tasks of ABEA (adapted from Zhang.2022b).
  • Figure 2: Overview of methodology.
  • Figure 3: Emotion labels in the ABEA training dataset.
  • Figure 4: Progression of model performance during 10-fold cross validation.
  • Figure 5: Left: Validation metrics per epoch, Right: Training metrics per epoch during when trying to adjust for overfitting (here showing hyperparameter configuration 3)
  • ...and 3 more figures