Event Causality Is Key to Computational Story Understanding

TL;DR

The paper addresses the gap in computational story understanding by leveraging event causality. It introduces a simple, prompt-based method to extract causal graphs from free-form narratives using large language models and demonstrates that these graphs align with human annotations and improve downstream tasks. Empirically, causal graphs yield state-of-the-art performance on COPES and meaningful gains in story quality evaluation and video-text alignment across diverse datasets, including OpenMEVA and SyMoN/YMS. The findings highlight substantial untapped potential for event causality in both text-only and multimodal story understanding and provide a publicly available codebase for reproducibility.

Abstract

Cognitive science and symbolic AI research suggest that event causality provides vital information for story understanding. However, machine learning systems for story understanding rarely employ event causality, partially due to the lack of methods that reliably identify open-world causal event relations. Leveraging recent progress in large language models, we present the first method for event causality identification that leads to material improvements in computational story understanding. Our technique sets a new state of the art on the COPES dataset (Wang et al., 2023) for causal event relation identification. Further, in the downstream story quality evaluation task, the identified causal relations lead to 3.6-16.6% relative improvement on correlation with human ratings. In the multimodal story video-text alignment task, we attain 4.1-10.9% increase on Clip Accuracy and 4.2-13.5% increase on Sentence IoU. The findings indicate substantial untapped potential for event causality in computational story understanding. The codebase is at https://github.com/insundaycathy/Event-Causality-Extraction.

Figures (14)

• Figure 1: The LLM prompt for event causal relationship extraction.
• Figure 2: Example of ChatGPT-generated event graph. The example is from GLUCOSE, the [Correct] labels are not part of the model output.
• Figure 3: The process of video context identification. The causal context is marked in green, the temporal context in purple, and the current item in red. For illustration purposes, the number of causal and temporal context items are both set to 2.
• Figure 4: The basic prompt for event causality
• Figure 5: The prompt instruction generated by ChatGPT