FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation

TL;DR

FreeSwim tackles the challenge of training-free ultra-high-resolution video generation by reusing pre-trained diffusion-transformer models at their native scales. It introduces an inward sliding-window attention mechanism to maintain the training-scale receptive field and a dual-path cross-attention override to fuse local detail with global coherence, supplemented by a cross-attention caching strategy for efficiency. Across Wan2.1 and LTX-Video, FreeSwim achieves state-of-the-art visual fidelity and semantic consistency at 4K–3K resolutions while delivering substantial speedups over full-attention baselines. The approach is validated through quantitative benchmarks, ablations, and a user study, demonstrating robust performance without additional data or training.

Abstract

The quadratic time and memory complexity of the attention mechanism in modern Transformer based video generators makes end-to-end training for ultra high resolution videos prohibitively expensive. Motivated by this limitation, we introduce a training-free approach that leverages video Diffusion Transformers pretrained at their native scale to synthesize higher resolution videos without any additional training or adaptation. At the core of our method lies an inward sliding window attention mechanism, which originates from a key observation: maintaining each query token's training scale receptive field is crucial for preserving visual fidelity and detail. However, naive local window attention, unfortunately, often leads to repetitive content and exhibits a lack of global coherence in the generated results. To overcome this challenge, we devise a dual-path pipeline that backs up window attention with a novel cross-attention override strategy, enabling the semantic content produced by local attention to be guided by another branch with a full receptive field and, therefore, ensuring holistic consistency. Furthermore, to improve efficiency, we incorporate a cross-attention caching strategy for this branch to avoid the frequent computation of full 3D attention. Extensive experiments demonstrate that our method delivers ultra-high-resolution videos with fine-grained visual details and high efficiency in a training-free paradigm. Meanwhile, it achieves superior performance on VBench, even compared to training-based alternatives, with competitive or improved efficiency. Codes are available at: https://github.com/WillWu111/FreeSwim

Figures (14)

• Figure 1: Ultra-resolution results generated by our FreeSwim built upon Wan2.1 wan2025wanopenadvancedlargescale. Resolution is marked on the top-right corner of each result in the format of width$\times$height. Corresponding prompts can be found in the appendix.
• Figure 2: Visual results and corresponding attention maps at $\times 2$ and $\times 4$ native resolutions from Lumina-Next gao2024luminat2xtransformingtextmodality and Wan2.1 wan2025wanopenadvancedlargescale, based on the same DiT architecture family.
• Figure 3: Qualitative comparison of generated results under different strategies based on Wan2.1 wan2025wanopenadvancedlargescale. Except for (a) and (b), all other results are produced using a coarse-to-fine scheme, where a base video (832 $\times$ 480) is first generated by the convention text-to-video pipeline, followed by high-resolution refinement (1920 $\times$ 1088) through SDEdit meng2021sdedit.
• Figure 4: FreeSwim Framework Overview. Upper Left: Our parallel Dual-Path pipeline combined with cross-attention override to achieve correct global semantic structure, with the Full-Branch Feature Reuse strategy in the figure representing the case where the cross attention of Full-Branch is computed and updated every two steps ($P$=2). Bottom Right: Our inward window attention ensures that during inference, the spatial dimension is strictly controlled at the same scale as during training.
• Figure 5: Visual comparison of synthesized 1080P videos for models based on Wan. Our method FreeSwim yields high-resolution videos characterized by high-fidelity details and coherent structure. Best viewed zoomed in. Corresponding prompts can be found in the appendix.