SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistency

TL;DR

SceneDecorator introduces a training-free framework for scene-oriented story generation, tackling scene planning and scene consistency. It combines VLM-Guided Scene Planning to decompose themes into global-to-local scenes and Long-Term Scene-Sharing Attention to preserve subject diversity and cross-story coherence, aided by Mask-Guided Scene Injection and Extrapolable Noise Blending. Across automatic metrics and user studies, it outperforms baselines in scene alignment and consistency, with ablations confirming the contribution of each component. The approach enables richer, more coherent visual storytelling with potential applications in arts, films, and games.

Abstract

Recent text-to-image models have revolutionized image generation, but they still struggle with maintaining concept consistency across generated images. While existing works focus on character consistency, they often overlook the crucial role of scenes in storytelling, which restricts their creativity in practice. This paper introduces scene-oriented story generation, addressing two key challenges: (i) scene planning, where current methods fail to ensure scene-level narrative coherence by relying solely on text descriptions, and (ii) scene consistency, which remains largely unexplored in terms of maintaining scene consistency across multiple stories. We propose SceneDecorator, a training-free framework that employs VLM-Guided Scene Planning to ensure narrative coherence across different scenes in a ``global-to-local'' manner, and Long-Term Scene-Sharing Attention to maintain long-term scene consistency and subject diversity across generated stories. Extensive experiments demonstrate the superior performance of SceneDecorator, highlighting its potential to unleash creativity in the fields of arts, films, and games.

Figures (11)

• Figure 1: Overview of SceneDecorator. SceneDecorator manages to "decorate" the scenes of story images, ensuring narrative coherence across different scenes (green arrow) and scene consistency across different stories (blue arrow), all based on a concise user-provided theme.
• Figure 2: Overall framework of SceneDecorator. (a) VLM-Guided Scene Planning involves conceptualizing, visualizing, and crafting in a "global-to-local" manner. (b) Long-Term Scene-Sharing Attention maintains long-range scene consistency and subject diversity across generated stories.
• Figure 3: Comparison of different methods. In (a), subject styles align with the scene but at the expense of diversity, whereas (b) better showcases diversity. Compared to (c), (d) further emphasizes scene consistency. Note that purple boxes highlight distinctions. Best viewed with zoom-in.
• Figure 4: Qualitative comparison of our SceneDecorator with other baselines. SceneDecorator demonstrates superior scene consistency and alignment across different stories compared to other baselines, making it well-suited for creative applications in filmmaking. Best viewed with zoom-in.
• Figure 5: Ablation study of the two components: Mask-Guided Scene Injection and Scene-Sharing Attention. " ✓" and " ✘" indicate whether each component is used. The synergy between these components ensures scene consistency and subject diversity across generated stories.