Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification

Alexander Shvets

TL;DR

The paper tackles the challenge of context in emotion classification by proposing Emo Pillars, a pipeline that uses Mistral-7b to generate diverse, narrative-grounded, context-rich and context-less emotion data. This data trains mid-sized encoders (e.g., RoBERTa-based) to perform robust context-aware and context-free emotion recognition, achieving state-of-the-art results on GoEmotions, ISEAR, and competitive performance on IEMOCAP and EmoContext. Key innovations include soft multi-label supervision, context-centric rewriting, and extensive human evaluation, which collectively improve utterance diversification and context personalization while highlighting limitations in neutral and out-of-taxonomy labels. The approach enables scalable, domain-adaptive emotion understanding with practical implications for real-world systems, such as analyzing user feedback in multimedia platforms, and points to future multilingual and explainability enhancements.

Abstract

Most datasets for sentiment analysis lack context in which an opinion was expressed, often crucial for emotion understanding, and are mainly limited by a few emotion categories. Foundation large language models (LLMs) like GPT-4 suffer from over-predicting emotions and are too resource-intensive. We design an LLM-based data synthesis pipeline and leverage a large model, Mistral-7b, for the generation of training examples for more accessible, lightweight BERT-type encoder models. We focus on enlarging the semantic diversity of examples and propose grounding the generation into a corpus of narratives to produce non-repetitive story-character-centered utterances with unique contexts over 28 emotion classes. By running 700K inferences in 450 GPU hours, we contribute with the dataset of 100K contextual and also 300K context-less examples to cover both scenarios. We use it for fine-tuning pre-trained encoders, which results in several Emo Pillars models. We show that Emo Pillars models are highly adaptive to new domains when tuned to specific tasks such as GoEmotions, ISEAR, IEMOCAP, and EmoContext, reaching the SOTA performance on the first three. We also validate our dataset, conducting statistical analysis and human evaluation, and confirm the success of our measures in utterance diversification (although less for the neutral class) and context personalization, while pointing out the need for improved handling of out-of-taxonomy labels within the pipeline.

Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification

TL;DR

The paper tackles the challenge of context in emotion classification by proposing Emo Pillars, a pipeline that uses Mistral-7b to generate diverse, narrative-grounded, context-rich and context-less emotion data. This data trains mid-sized encoders (e.g., RoBERTa-based) to perform robust context-aware and context-free emotion recognition, achieving state-of-the-art results on GoEmotions, ISEAR, and competitive performance on IEMOCAP and EmoContext. Key innovations include soft multi-label supervision, context-centric rewriting, and extensive human evaluation, which collectively improve utterance diversification and context personalization while highlighting limitations in neutral and out-of-taxonomy labels. The approach enables scalable, domain-adaptive emotion understanding with practical implications for real-world systems, such as analyzing user feedback in multimedia platforms, and points to future multilingual and explainability enhancements.

Abstract

Most datasets for sentiment analysis lack context in which an opinion was expressed, often crucial for emotion understanding, and are mainly limited by a few emotion categories. Foundation large language models (LLMs) like GPT-4 suffer from over-predicting emotions and are too resource-intensive. We design an LLM-based data synthesis pipeline and leverage a large model, Mistral-7b, for the generation of training examples for more accessible, lightweight BERT-type encoder models. We focus on enlarging the semantic diversity of examples and propose grounding the generation into a corpus of narratives to produce non-repetitive story-character-centered utterances with unique contexts over 28 emotion classes. By running 700K inferences in 450 GPU hours, we contribute with the dataset of 100K contextual and also 300K context-less examples to cover both scenarios. We use it for fine-tuning pre-trained encoders, which results in several Emo Pillars models. We show that Emo Pillars models are highly adaptive to new domains when tuned to specific tasks such as GoEmotions, ISEAR, IEMOCAP, and EmoContext, reaching the SOTA performance on the first three. We also validate our dataset, conducting statistical analysis and human evaluation, and confirm the success of our measures in utterance diversification (although less for the neutral class) and context personalization, while pointing out the need for improved handling of out-of-taxonomy labels within the pipeline.

Paper Structure

This paper contains 42 sections, 10 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Synthesis Pipeline
  4. Content-rich instructions
  5. Multiple example generation
  6. Soft labelling
  7. Context generation and cleaning
  8. Context importance upscale
  9. Dataset Generation
  10. Experiments
  11. Context-less emotion recognition
  12. Emo Pillars synthetic test sets
  13. GoEmotions
  14. ISEAR
  15. IEMOCAP
  16. ...and 27 more sections

Figures (10)

  • Figure 1: Difference in context-less (context cannot be taken into account) and context-aware (context helps) emotion classification. Context-less models detect emotions in the entire input (including context if provided), while context-aware models can grasp the input structure and extract emotions only from the utterance.
  • Figure 2: Our pipeline for the generation of a dataset for a multi-label context-aware (-less) emotion recognition.
  • Figure 3: Distribution of primary emotions.
  • Figure 4: Distribution of soft emotional labels in the dataset (after filtering by the expressiveness level).
  • Figure 5: Dataset splits. $Orig$ -- context-less examples, $COrig$ -- context-full examples, $CRewr$ -- the same context-full examples with rewritten utterances.
  • ...and 5 more figures