HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives

TL;DR

HoloCine tackles the challenge of generating coherent, multi-shot cinematic narratives from text prompts by co-designing a holistic generation framework with two specialized mechanisms: Window Cross-Attention for per-shot directive control and Sparse Inter-Shot Self-Attention for scalable, cross-shot coherence. A large-scale hierarchical data pipeline (400k samples) supports training a DiT-based diffusion model to produce entire scenes in a single pass, achieving state-of-the-art performance in transition control, inter-shot and intra-shot consistency, and semantic fidelity. The approach reveals emergent capabilities, including persistent memory for characters and a nuanced grasp of cinematic language, while enabling precise cinematographic commands such as shot scale, angles, and movement. Limitations remain in causal reasoning, pointing to future work on action-to-state understanding, but HoloCine marks a major step toward end-to-end automated filmmaking from text prompts.

Abstract

State-of-the-art text-to-video models excel at generating isolated clips but fall short of creating the coherent, multi-shot narratives, which are the essence of storytelling. We bridge this "narrative gap" with HoloCine, a model that generates entire scenes holistically to ensure global consistency from the first shot to the last. Our architecture achieves precise directorial control through a Window Cross-Attention mechanism that localizes text prompts to specific shots, while a Sparse Inter-Shot Self-Attention pattern (dense within shots but sparse between them) ensures the efficiency required for minute-scale generation. Beyond setting a new state-of-the-art in narrative coherence, HoloCine develops remarkable emergent abilities: a persistent memory for characters and scenes, and an intuitive grasp of cinematic techniques. Our work marks a pivotal shift from clip synthesis towards automated filmmaking, making end-to-end cinematic creation a tangible future. Our code is available at: https://holo-cine.github.io/.

Figures (7)

• Figure 1: The architecture of our holistic generation pipeline, where all shot latents are processed jointly. The Window Cross-Attention provides precise directorial control by aligning each shot to its specific text prompt. The Sparse Inter-shot Self-Attention drastically reduces computational cost while preserving long-range consistency.
• Figure 2: Qualitative comparison on a complex multi-shot prompt. Our method successfully generates a coherent sequence of distinct shots aligned with per-shot descriptions, while baseline methods fail in maintaining consistency, prompt fidelity, or handling shot transitions.
• Figure 3: Qualitative comparison with state-of-the-art commercial models. While Vidu and Kling 2.5 Turbo fail to interpret multi-shot instructions and generate only a single, continuous clip, HoloCine successfully executes complex shot transitions. Our method demonstrates narrative control and consistency comparable to the leading closed-source model, Sora 2, accurately rendering the sequence from medium shots to close-ups as directed by the prompt.
• Figure 4: Qualitative results for our ablation study. We compare our full model (bottom row) against three variants. From top to bottom: removing window cross-attention prevents shot transitions; a full self-attention baseline works well but is computationally expensive; and removing inter-shot summary tokens leads to a complete loss of character consistency.
• Figure 5: Qualitative results of our model's emergent memory capability. (a) Character Permanence: The subject's identity and appearance are consistently maintained across different camera angles and expressions. (b) Long-range Consistency: The subject is accurately recalled after an unrelated distractor shot. (c) Fine-grained Persistence: A non-salient background detail (the magnet, highlighted) is correctly preserved across an intervening shot.