Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NarrativePlay: Interactive Narrative Understanding

Runcong Zhao, Wenjia Zhang, Jiazheng Li, Lixing Zhu, Yanran Li, Yulan He, Lin Gui

TL;DR

NarrativePlay tackles the challenge of turning long-form narratives into interactive, character-driven experiences by extracting structured storyline elements and synchronizing text with visuals and speech. The approach uses a modular architecture with memory-augmented prompts, stable diffusion for environment visualization, and TTS for dialogue, enabling a non-sandboxed, perspective-driven exploration of stories. The paper contributes an end-to-end extraction and progression pipeline, a multimodal rendering framework, and an evaluation framework across adventure and detective narratives to assess coherence, relevance, empathy, and commonsense. Results show memory enhances relevance and commonsense but reveal gaps in coherence and empathy, especially in complex plots, guiding future improvements in non-linear timelines and content quality.

Abstract

In this paper, we introduce NarrativePlay, a novel system that allows users to role-play a fictional character and interact with other characters in narratives such as novels in an immersive environment. We leverage Large Language Models (LLMs) to generate human-like responses, guided by personality traits extracted from narratives. The system incorporates auto-generated visual display of narrative settings, character portraits, and character speech, greatly enhancing user experience. Our approach eschews predefined sandboxes, focusing instead on main storyline events extracted from narratives from the perspective of a user-selected character. NarrativePlay has been evaluated on two types of narratives, detective and adventure stories, where users can either explore the world or improve their favorability with the narrative characters through conversations.

NarrativePlay: Interactive Narrative Understanding

TL;DR

NarrativePlay tackles the challenge of turning long-form narratives into interactive, character-driven experiences by extracting structured storyline elements and synchronizing text with visuals and speech. The approach uses a modular architecture with memory-augmented prompts, stable diffusion for environment visualization, and TTS for dialogue, enabling a non-sandboxed, perspective-driven exploration of stories. The paper contributes an end-to-end extraction and progression pipeline, a multimodal rendering framework, and an evaluation framework across adventure and detective narratives to assess coherence, relevance, empathy, and commonsense. Results show memory enhances relevance and commonsense but reveal gaps in coherence and empathy, especially in complex plots, guiding future improvements in non-linear timelines and content quality.

Abstract

In this paper, we introduce NarrativePlay, a novel system that allows users to role-play a fictional character and interact with other characters in narratives such as novels in an immersive environment. We leverage Large Language Models (LLMs) to generate human-like responses, guided by personality traits extracted from narratives. The system incorporates auto-generated visual display of narrative settings, character portraits, and character speech, greatly enhancing user experience. Our approach eschews predefined sandboxes, focusing instead on main storyline events extracted from narratives from the perspective of a user-selected character. NarrativePlay has been evaluated on two types of narratives, detective and adventure stories, where users can either explore the world or improve their favorability with the narrative characters through conversations.
Paper Structure (36 sections, 2 figures, 4 tables)

This paper contains 36 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Architecture of NarrativePlay
  3. Main Storyline Extraction
  4. Character
  5. Events
  6. Environments
  7. Narrative Image and Speech Synthesis
  8. Narrative Image Synthesis
  9. Narrative Speech Synthesis
  10. Main Storyline Progression
  11. Narrative Input
  12. Character Selection
  13. Story Progression
  14. Evaluation
  15. Evaluation Schema
  16. ...and 21 more sections

Figures (2)

  • Figure 1: Our system's interactive process begins when a user provides a narrative to the system. They then choose a character as their narrative identity, through whom they can engage with the story. Users can have conversations with other characters, thereby experiencing the story in a more immersive way.
  • Figure 2: Demonstration of our NarrativePlay through a pipeline view.