Understanding Emotion in Discourse: Recognition Insights and Linguistic Patterns for Generation

TL;DR

This work tackles two gaps in emotion recognition in conversation (ERC): identifying which architectural choices truly influence performance and connecting recognition to generation through linguistic analysis. Using a rigorous 10-seed evaluation on IEMOCAP and a large-scale discourse-marker study, the authors show that conversational context is the dominant factor, while hierarchical utterance representations provide limited gains once context is available, and external lexicons like SenticNet offer no additional benefit. A detailed analysis of discourse markers reveals emotion-specific patterns, notably reduced left-periphery usage for sadness, which helps explain its strong reliance on conversational history. The results demonstrate strong performance with strictly causal (past-only) context, offer actionable generation-oriented insights, and raise questions about the validity of the 6-way taxonomy for text-only ERC, guiding future research toward context-aware architectures and linguistically grounded generation guidelines.

Abstract

While Emotion Recognition in Conversation (ERC) has achieved high accuracy, two critical gaps remain: a limited understanding of \textit{which} architectural choices actually matter, and a lack of linguistic analysis connecting recognition to generation. We address both gaps through a systematic analysis of the IEMOCAP dataset. For recognition, we conduct a rigorous ablation study with 10-seed evaluation and report three key findings. First, conversational context is paramount, with performance saturating rapidly -- 90\% of the total gain achieved within just the most recent 10--30 preceding turns (depending on the label set). Second, hierarchical sentence representations help at utterance-level, but this benefit disappears once conversational context is provided, suggesting that context subsumes intra-utterance structure. Third, external affective lexicons (SenticNet) provide no gain, indicating that pre-trained encoders already capture necessary emotional semantics. With simple architectures using strictly causal context, we achieve 82.69\% (4-way) and 67.07\% (6-way) weighted F1, outperforming prior text-only methods including those using bidirectional context. For linguistic analysis, we analyze 5,286 discourse marker occurrences and find a significant association between emotion and marker positioning ($p < .0001$). Notably, "sad" utterances exhibit reduced left-periphery marker usage (21.9\%) compared to other emotions (28--32\%), consistent with theories linking left-periphery markers to active discourse management. This connects to our recognition finding that sadness benefits most from context (+22\%p): lacking explicit pragmatic signals, sad utterances require conversational history for disambiguation.

Figures (3)

• Figure 1: IEMOCAP dataset characteristics.
• Figure 2: Model architecture. Each turn is encoded independently via Sentence-RoBERTa using either flat (whole utterance) or hierarchical (sentence-level) encoding. SenticNet features are optionally fused (dashed boxes). For classification, we use MLP when K=0 (no context) or unidirectional LSTM when K>0 (with preceding turns as context).
• Figure 3: Per-emotion F1 scores across context sizes (4-way classification). Angry saturates quickly at $K^*=10$ (10 preceding turns), while Sad, Happy, and Neutral require extended context ($K^*=130$--$140$ preceding turns). Shaded regions indicate $\pm 1$ standard deviation across 10 seeds.