The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation

TL;DR

This work tackles the challenge of generating long-form, coherent cinematic videos from sparse dialogue by proposing an end-to-end agentic pipeline. It introduces ScripterAgent to translate dialogue into executable cinematic scripts, DirectorAgent to maintain long-horizon visual coherence via cross-scene generation, and CriticAgent for comprehensive evaluation, all trained and validated on ScriptBench, a new large-scale cinematic script benchmark. A two-stage training regime for ScripterAgent—supervised fine-tuning followed by Group Relative Policy Optimization with a hybrid reward—bridges structural accuracy and aesthetic quality, while the DirectorAgent employs frame-anchored continuity to reduce identity drift across scenes. The framework yields measurable gains in script faithfulness and temporal fidelity, demonstrates a trade-off between visual spectacle and script adherence among SOTA video models, and introduces Visual-Script Alignment as a robust metric for temporal-semantic coherence. This approach advances automated filmmaking by enabling end-to-end dialogue-driven storytelling and provides practical guidance for selecting models based on whether visual realism or narrative fidelity is prioritized.

Abstract

Recent advances in video generation have produced models capable of synthesizing stunning visual content from simple text prompts. However, these models struggle to generate long-form, coherent narratives from high-level concepts like dialogue, revealing a ``semantic gap'' between a creative idea and its cinematic execution. To bridge this gap, we introduce a novel, end-to-end agentic framework for dialogue-to-cinematic-video generation. Central to our framework is ScripterAgent, a model trained to translate coarse dialogue into a fine-grained, executable cinematic script. To enable this, we construct ScriptBench, a new large-scale benchmark with rich multimodal context, annotated via an expert-guided pipeline. The generated script then guides DirectorAgent, which orchestrates state-of-the-art video models using a cross-scene continuous generation strategy to ensure long-horizon coherence. Our comprehensive evaluation, featuring an AI-powered CriticAgent and a new Visual-Script Alignment (VSA) metric, shows our framework significantly improves script faithfulness and temporal fidelity across all tested video models. Furthermore, our analysis uncovers a crucial trade-off in current SOTA models between visual spectacle and strict script adherence, providing valuable insights for the future of automated filmmaking.

Figures (5)

• Figure 1: Our proposed pipeline consists of three key components: (1) a ScripterAgent, trained with GRPO to align its outputs with professional directorial standards; (2) a DirectorAgent, which ensures seamless visual continuity across scenes, thereby overcoming the temporal incoherence caused by the fixed-duration constraints of SOTA video generation models; and (3) a CriticAgent, which evaluates the generated film from both technical and cinematic perspectives.
• Figure 2: The three-stage, expert-guided pipeline for creating the ScriptBench.
• Figure 3: Illustration of the Frame-Anchoring Mechanism. The DirectorAgent utilizes the last frame of the preceding scene ($N-1$) to visually condition the generation of the current scene's first frame (0). This frame-anchoring strategy preserves visual consistency in character appearance, attire, and scene layout across multiple generation cycles, creating a seamless visual relay.
• Figure 4: Qualitative comparison: ScripterAgent vs. MovieAgent wu2025automated.
• Figure 5: Examples of video generation using Sora2-Pro.