Disambiguation of Emotion Annotations by Contextualizing Events in Plausible Narratives

Abstract

Ambiguity in emotion analysis stems both from potentially missing information and the subjectivity of interpreting a text. The latter did receive substantial attention, but can we fill missing information to resolve ambiguity? We address this question by developing a method to automatically generate reasonable contexts for an otherwise ambiguous classification instance. These generated contexts may act as illustrations of potential interpretations by different readers, as they can fill missing information with their individual world knowledge. This task to generate plausible narratives is a challenging one: We combine techniques from short story generation to achieve coherent narratives. The resulting English dataset of Emotional BackStories, EBS, allows for the first comprehensive and systematic examination of contextualized emotion analysis. We conduct automatic and human annotation and find that the generated contextual narratives do indeed clarify the interpretation of specific emotions. Particularly relief and sadness benefit from our approach, while joy does not require the additional context we provide.

Figures (4)

• Figure 1: Overview of our LLM-based data generation framework for event chains according to three different methods. The prompts used in each case are shown summarized -- for full text prompts see \ref{['app:prompts']}.
• Figure 2: Emotion annotation agreement on events in comparison to Baseline and PCR chains.
• Figure 3: Mean of trajectories of predicted emotion probability scores in event chains for selected emotions ($\{s_1, ..., s_5\}$, generation method: PCR).
• Figure 4: Mean and standard deviation of emotion trajectories in event chains (x-axes corresponding to the first n sentences of the chains; generation method: PCR) in comparison to mean emotion of events in isolation (dashed lines).