Tuning-Free Multi-Event Long Video Generation via Synchronized Coupled Sampling

TL;DR

This work tackles long, multi-event video generation with diffusion models by removing the need for additional tuning. It introduces SynCoS, a three-stage, synchronized sampling framework that jointly enforces local smoothness via DDIM-based fusion and global coherence via Collaborative Score Distillation, tightly coupled with a grounded timestep and fixed baseline noise. A structured prompt—combining a global scenario with local event-specific prompts—guides coherent dynamics across segments. Across 48 challenging long-video scenarios and multiple backbones, SynCoS delivers superior temporal consistency, high per-frame quality, and strong prompt fidelity, albeit with increased inference time. The approach is architecture-agnostic and demonstrates substantial practical impact for scalable, high-quality long video synthesis without retraining.

Abstract

While recent advancements in text-to-video diffusion models enable high-quality short video generation from a single prompt, generating real-world long videos in a single pass remains challenging due to limited data and high computational costs. To address this, several works propose tuning-free approaches, i.e., extending existing models for long video generation, specifically using multiple prompts to allow for dynamic and controlled content changes. However, these methods primarily focus on ensuring smooth transitions between adjacent frames, often leading to content drift and a gradual loss of semantic coherence over longer sequences. To tackle such an issue, we propose Synchronized Coupled Sampling (SynCoS), a novel inference framework that synchronizes denoising paths across the entire video, ensuring long-range consistency across both adjacent and distant frames. Our approach combines two complementary sampling strategies: reverse and optimization-based sampling, which ensure seamless local transitions and enforce global coherence, respectively. However, directly alternating between these samplings misaligns denoising trajectories, disrupting prompt guidance and introducing unintended content changes as they operate independently. To resolve this, SynCoS synchronizes them through a grounded timestep and a fixed baseline noise, ensuring fully coupled sampling with aligned denoising paths. Extensive experiments show that SynCoS significantly improves multi-event long video generation, achieving smoother transitions and superior long-range coherence, outperforming previous approaches both quantitatively and qualitatively.

Figures (15)

• Figure 1: Quantitative comparison. Experimental results of SynCoS with baselines on multi-event long video generations. Bold indicates the best results. SynCoS consistently outperforms baselines across temporal consistency, per-frame quality, and prompt fidelity.
• Figure 2: Qualitative comparisons on CogVideoX-2B yang2024cogvideox. All examples are 5 times longer in duration compared to the underlying base model, generating a 30-second video. Unlike Gen-L-Video wang2023gen and FIFO-Diffusion kim2024fifo, which often struggle with overlapping artifacts and style drift, our method, SynCoS, ensures consistency in both content and style throughout the entire video. Additionally, SynCoS generates long videos where each frame faithfully follows its designated prompt while ensuring seamless transitions between frames.
• Figure 2: Quantitative ablation study. *Abbreviations: subject consistency (SC), background consistency (BC), aesthetic quality (AQ), and prompt fidelity (PF).
• Figure 3: t-SNE visualization of CLIP radford2021learning features for the predicted video frames $\hat{\mathbf{x}}_{0|t}$, at each timestep using different samplings. Faded colors indicate earlier timesteps ($t \approx 1000$), while vivid colors indicate later, small timesteps ($t \approx 0$), illustrating feature trajectory evolution over time (top to bottom).
• Figure 3: Quantitative ablations study of the three coupled stages in SynCoS, omitting each stage during one-timestep denoising, demonstrates the importance of all three stages for coherent long video generation with multiple events.