Enhancing Event Extraction from Short Stories through Contextualized Prompts

TL;DR

The paper tackles event extraction in literary text by introducing Vrittanta-EN, a 1000-story corpus of Indian short stories annotated for real events across seven classes, and a contextualized-prompt framework using BART to detect and classify event triggers and spans. It demonstrates that prompt-based learning outperforms traditional neural baselines, achieving an event-trigger F1 of 89.9% and notable gains for the CONFLICT class in an imbalanced dataset. The authors also explore data augmentation to 1000 stories, showing that large quantities of synthetic labels can compensate for limited high-quality annotations, and provide thorough error analysis to guide future improvements. Overall, the work advances domain-specific resources for literary NLP and highlights the efficacy of contextualized prompts for low-resource, imbalanced information extraction tasks.

Abstract

Event extraction is an important natural language processing (NLP) task of identifying events in an unstructured text. Although a plethora of works deal with event extraction from new articles, clinical text etc., only a few works focus on event extraction from literary content. Detecting events in short stories presents several challenges to current systems, encompassing a different distribution of events as compared to other domains and the portrayal of diverse emotional conditions. This paper presents \texttt{Vrittanta-EN}, a collection of 1000 English short stories annotated for real events. Exploring this field could result in the creation of techniques and resources that support literary scholars in improving their effectiveness. This could simultaneously influence the field of Natural Language Processing. Our objective is to clarify the intricate idea of events in the context of short stories. Towards the objective, we collected 1,000 short stories written mostly for children in the Indian context. Further, we present fresh guidelines for annotating event mentions and their categories, organized into \textit{seven distinct classes}. The classes are {\tt{COGNITIVE-MENTAL-STATE(CMS), COMMUNICATION(COM), CONFLICT(CON), GENERAL-ACTIVITY(GA), LIFE-EVENT(LE), MOVEMENT(MOV), and OTHERS(OTH)}}. Subsequently, we apply these guidelines to annotate the short story dataset. Later, we apply the baseline methods for automatically detecting and categorizing events. We also propose a prompt-based method for event detection and classification. The proposed method outperforms the baselines, while having significant improvement of more than 4\% for the class \texttt{CONFLICT} in event classification task.

Figures (8)

• Figure 1: Examples of event triggers in News, bio-medical and short stories. News and Biomedical domains generally deal with real-life scenarios, while short stories may contain real-life scenarios in unreal situations.
• Figure 2: Distribution of different events in the dataset
• Figure 3: Distribution of events from different sources in the dataset
• Figure 4: Word cloud of un-annotated corpus and trigger words of the entire dataset
• Figure 5: Representation of the prompt-based architecture for event extraction and classification. $\circ$ denotes element-wise multiplication, $\times$ denotes matrix multiplication. Underlined tokens are predicted when the corresponding input is being used.Yellow represents that weights are updated in the layer while Blue represents that weights are not updated in the layer.