PetKaz at SemEval-2024 Task 3: Advancing Emotion Classification with an LLM for Emotion-Cause Pair Extraction in Conversations

TL;DR

This work tackles emotion-cause pair extraction in conversations (ECPE) by a two-stage approach: first, emotions are classified using a fine-tuned GPT-3.5 model, and second, causes are detected with a BiLSTM-based network that leverages contextual utterance representations. The system processes dialogue history to identify causal antecedents for non-neutral emotions, outputting labelled pairs $(U_i, U_t, E_t)$ without explicit span extraction due to dataset issues. On the SemEval-2024 Task 3 Subtask 1 dataset (Friends dialogues), the method achieves a weighted-average proportional $F_1$ of $0.264$ and ranks 2nd out of 15 teams, with a strict $F_1$ of $0.104$. The study analyzes emotion classification strengths and the challenges of cause detection, highlighting data- and annotation-related complexities and proposing future improvements such as better data annotation and enhanced speaker representations.

Abstract

In this paper, we present our submission to the SemEval-2023 Task~3 "The Competition of Multimodal Emotion Cause Analysis in Conversations", focusing on extracting emotion-cause pairs from dialogs. Specifically, our approach relies on combining fine-tuned GPT-3.5 for emotion classification and a BiLSTM-based neural network to detect causes. We score 2nd in the ranking for Subtask 1, demonstrating the effectiveness of our approach through one of the highest weighted-average proportional F1 scores recorded at 0.264.

Figures (7)

• Figure 1: The pipeline for ECPE. Utterances are classified with emotion labels $E_i$, and speakers are represented with one-hot vectors $S_i$. Utterances are then encoded with pre-trained sentence embeddings $U_i$ and enriched with context by BiLSTM $\bar{U}_i$. For each target utterance $U_t$, we detect whether any other utterance from the conversation history $H(U_t)$ is causal using a feed-forward network. $\bar{U}_i$, $S_i$ (of a potential causal utterance), $\bar{U}_t$, $S_t$, and $E_t$ are concatenated, and then, binary classification is performed. The pipeline outputs labelled emotion-cause pairs $(U_i, U_t, E_t)$.
• Figure 2: The prompt used to perform emotion classification with GPT-3.5.
• Figure 3: Performance of our emotion classifier on our development set.
• Figure 4: An example of a dialog where our model classified neutral utterance as joy.
• Figure 5: Performance of the cause extractor.