EmoStory: Emotion-Aware Story Generation

TL;DR

EmoStory is introduced, a two-stage framework that integrates agent-based story planning and region-aware story generation that outperforms state-of-the-art story generation methods in emotion accuracy, prompt alignment, and subject consistency.

Abstract

Story generation aims to produce image sequences that depict coherent narratives while maintaining subject consistency across frames. Although existing methods have excelled in producing coherent and expressive stories, they remain largely emotion-neutral, focusing on what subject appears in a story while overlooking how emotions shape narrative interpretation and visual presentation. As stories are intended to engage audiences emotionally, we introduce emotion-aware story generation, a new task that aims to generate subject-consistent visual stories with explicit emotional directions. This task is challenging due to the abstract nature of emotions, which must be grounded in concrete visual elements and consistently expressed across a narrative through visual composition. To address these challenges, we propose EmoStory, a two-stage framework that integrates agent-based story planning and region-aware story generation. The planning stage transforms target emotions into coherent story prompts with emotion agent and writer agent, while the generation stage preserves subject consistency and injects emotion-related elements through region-aware composition. We evaluate EmoStory on a newly constructed dataset covering 25 subjects and 600 emotional stories. Extensive quantitative and qualitative results, along with user studies, show that EmoStory outperforms state-of-the-art story generation methods in emotion accuracy, prompt alignment, and subject consistency.

Figures (5)

• Figure 1: Emotion-aware story generation with EmoStory, which introduces emotions (top: positive, bottom: negative) to given subjects (middle: neutral), generating coherent and emotionally expressive visual stories.
• Figure 2: Overview of EmoStory. Agent-based story planning maps abstract emotions to concrete emotional prompts at semantic level, while region-aware story generation preserves subject consistency and enhances emotional expressiveness at pixel level.
• Figure 3: Comparisons with state-of-the-art methods. EmoStory is superior in both emotion evocation and story expressiveness.
• Figure 4: EmoStory (red) outperforms all compared methods (blue) and ablations (green) across three evaluation metrics.
• Figure 5: Visualize the evolution of subject attention masks under two settings: w/o, w region disentanglement (RD).