STAGE: A Benchmark for Knowledge Graph Construction, Question Answering, and In-Script Role-Playing over Movie Screenplays

TL;DR

STAGE presents a unified benchmark for narrative understanding over full-length movie screenplays by modeling each screenplay as a coherent narrative world and evaluating four interconnected tasks: knowledge-graph construction (STAGE-KG), scene-level event summarization (STAGE-ES), long-context screenplay question answering (STAGE-QA), and in-script character role-playing (STAGE-ICRP). The dataset provides cleaned scripts, a canonical knowledge graph, and event- and character-centric annotations for 150 films in English and Chinese, enabling holistic evaluation of world modeling, abstraction, cross-scene reasoning, and faithful character generation. Experimental results across multiple backbones show that scalable models, memory-augmented memories, and explicit narrative facts improve coherence and grounding, though challenges remain in causal reasoning, temporal dynamics, and multilingual coverage. By integrating world modeling with memory-grounded generation, STAGE offers a scalable, multi-task platform to push toward truly consistent long-form narrative understanding and generation.

Abstract

Movie screenplays are rich long-form narratives that interleave complex character relationships, temporally ordered events, and dialogue-driven interactions. While prior benchmarks target individual subtasks such as question answering or dialogue generation, they rarely evaluate whether models can construct a coherent story world and use it consistently across multiple forms of reasoning and generation. We introduce STAGE (Screenplay Text, Agents, Graphs and Evaluation), a unified benchmark for narrative understanding over full-length movie screenplays. STAGE defines four tasks: knowledge graph construction, scene-level event summarization, long-context screenplay question answering, and in-script character role-playing, all grounded in a shared narrative world representation. The benchmark provides cleaned scripts, curated knowledge graphs, and event- and character-centric annotations for 150 films across English and Chinese, enabling holistic evaluation of models' abilities to build world representations, abstract and verify narrative events, reason over long narratives, and generate character-consistent responses.

Figures (20)

• Figure 1: Overview of the STAGE benchmark. Given a screenplay as a coherent narrative world, STAGE evaluates four complementary capabilities: (1) constructing a movie-level knowledge graph of stable entities and relations; (2) summarizing scene-level events and recovering their factual content; (3) answering long-context screenplay questions requiring cross-scene reasoning; and (4) in-script character role-playing under narrative and factual constraints.
• Figure 2: Pipeline for constructing narrative knowledge graphs in STAGE. Screenplay text is processed through staged extraction, with narratively salient events identified first, followed by associated entities and typed relations under schema constraints. Rule-based refinement and normalization consolidate the graph into a coherent, movie-level representation grounded in the original screenplay.
• Figure 3: Overview of the STAGE-ICRP annotation pipeline. Character-centric action, dialogues, and narrative facts are extracted from the screenplay and used to construct persona cards, select dialogue exemplars, and define interaction constraints.
• Figure 4: Word cloud visualization of the consolidated genre distribution in STAGE. The size of each genre label is proportional to its frequency aggregated across all genre annotation slots.
• Figure 5: Distribution of screenplay lengths and scene counts in STAGE. The left histogram shows the distribution of screenplay word counts, while the right histogram shows the distribution of scene counts across all movies in the dataset.