Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReDiStory: Region-Disentangled Diffusion for Consistent Visual Story Generation

Ayushman Sarkar, Zhenyu Yu, Chu Chen, Wei Tang, Kangning Cui, Mohd Yamani Idna Idris

TL;DR

ReDiStory tackles identity drift in multi-frame visual storytelling by introducing a training-free, inference-time prompt embedding reorganization that decouples identity from frame-specific content. By explicitly splitting embeddings into identity-related and frame-specific components and decorrelating cross-frame directions, it reduces inter-frame interference without modifying the diffusion backbone or requiring extra supervision. Evaluations on the ConsiStory+ benchmark show consistent identity-consistency gains with only modest increases in memory and latency. The approach is lightweight, model-agnostic, and promises scalable improvements for long-form visual narratives.

Abstract

Generating coherent visual stories requires maintaining subject identity across multiple images while preserving frame-specific semantics. Recent training-free methods concatenate identity and frame prompts into a unified representation, but this often introduces inter-frame semantic interference that weakens identity preservation in complex stories. We propose ReDiStory, a training-free framework that improves multi-frame story generation via inference-time prompt embedding reorganization. ReDiStory explicitly decomposes text embeddings into identity-related and frame-specific components, then decorrelates frame embeddings by suppressing shared directions across frames. This reduces cross-frame interference without modifying diffusion parameters or requiring additional supervision. Under identical diffusion backbones and inference settings, ReDiStory improves identity consistency while maintaining prompt fidelity. Experiments on the ConsiStory+ benchmark show consistent gains over 1Prompt1Story on multiple identity consistency metrics. Code is available at: https://github.com/YuZhenyuLindy/ReDiStory

ReDiStory: Region-Disentangled Diffusion for Consistent Visual Story Generation

TL;DR

ReDiStory tackles identity drift in multi-frame visual storytelling by introducing a training-free, inference-time prompt embedding reorganization that decouples identity from frame-specific content. By explicitly splitting embeddings into identity-related and frame-specific components and decorrelating cross-frame directions, it reduces inter-frame interference without modifying the diffusion backbone or requiring extra supervision. Evaluations on the ConsiStory+ benchmark show consistent identity-consistency gains with only modest increases in memory and latency. The approach is lightweight, model-agnostic, and promises scalable improvements for long-form visual narratives.

Abstract

Generating coherent visual stories requires maintaining subject identity across multiple images while preserving frame-specific semantics. Recent training-free methods concatenate identity and frame prompts into a unified representation, but this often introduces inter-frame semantic interference that weakens identity preservation in complex stories. We propose ReDiStory, a training-free framework that improves multi-frame story generation via inference-time prompt embedding reorganization. ReDiStory explicitly decomposes text embeddings into identity-related and frame-specific components, then decorrelates frame embeddings by suppressing shared directions across frames. This reduces cross-frame interference without modifying diffusion parameters or requiring additional supervision. Under identical diffusion backbones and inference settings, ReDiStory improves identity consistency while maintaining prompt fidelity. Experiments on the ConsiStory+ benchmark show consistent gains over 1Prompt1Story on multiple identity consistency metrics. Code is available at: https://github.com/YuZhenyuLindy/ReDiStory
Paper Structure (22 sections, 6 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 22 sections, 6 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Subject-Consistent Text-to-Image Generation.
  4. Training-Free Multi-Frame Storytelling.
  5. Prompt Structure and Semantic Interference.
  6. Method
  7. Problem Formulation
  8. Prompt Embedding Representation
  9. Prompt Embedding Reorganization
  10. Generation and Computational Complexity
  11. Experiments
  12. Experimental Setup
  13. Dataset
  14. Baselines
  15. Implementation details
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Motivation. Identity-related components (e.g., name and appearance) can be entangled with frame-specific instructions (e.g., location and pose), leading to unintended identity changes across frames. We therefore separate these two components for more consistent multi-frame generation.
  • Figure 2: ReDiStory pipeline. Identity and frame prompts are jointly encoded, decomposed at the token level, and reorganized by projection-based decorrelation to reduce inter-frame semantic interference before standard diffusion sampling.
  • Figure 3: Qualitative comparison on multi-frame generation. Under identical identity and frame prompts, ReDiStory preserves subject-specific appearance as the scene and pose vary.
  • Figure A.1: Additional comparison results.