UltraGen: High-Resolution Video Generation with Hierarchical Attention

TL;DR

UltraGen tackles the challenge of native high-resolution video generation by decoupling full attention into time-aware global and local branches, augmented with spatially compressed global attention and cross-window hierarchical local attention. This hierarchical dual-branch framework enables end-to-end 1080P–4K synthesis with significantly reduced computation, outperforming state-of-the-art and two-stage pipelines in quality and speed. The method includes domain-aware LoRA adaptations and a dynamic fusion mechanism that balances global coherence with local detail across timesteps. Experiments demonstrate strong quantitative and qualitative gains, marking a practical advance in HD video generation from pre-trained low-resolution diffusion models.

Abstract

Recent advances in video generation have made it possible to produce visually compelling videos, with wide-ranging applications in content creation, entertainment, and virtual reality. However, most existing diffusion transformer based video generation models are limited to low-resolution outputs (<=720P) due to the quadratic computational complexity of the attention mechanism with respect to the output width and height. This computational bottleneck makes native high-resolution video generation (1080P/2K/4K) impractical for both training and inference. To address this challenge, we present UltraGen, a novel video generation framework that enables i) efficient and ii) end-to-end native high-resolution video synthesis. Specifically, UltraGen features a hierarchical dual-branch attention architecture based on global-local attention decomposition, which decouples full attention into a local attention branch for high-fidelity regional content and a global attention branch for overall semantic consistency. We further propose a spatially compressed global modeling strategy to efficiently learn global dependencies, and a hierarchical cross-window local attention mechanism to reduce computational costs while enhancing information flow across different local windows. Extensive experiments demonstrate that UltraGen can effectively scale pre-trained low-resolution video models to 1080P and even 4K resolution for the first time, outperforming existing state-of-the-art methods and super-resolution based two-stage pipelines in both qualitative and quantitative evaluations.

Figures (9)

• Figure 1: Typical video generation models exhibit significant quality degradation and increased processing time with higher resolutions, whereas our UltraGen delivers superior video quality at resolutions beyond 2K while achieving 4.78$\times$ speedup compared to the popular Wan-T2V-1.3B baseline wang2025wan (81 frames, 4$\times$H20 GPUs). Enlarge for better visual effects.
• Figure 2: Overview of our UltraGen that decomposes the full-attention into a global attention branch (Sec. \ref{['sec:global']}) for overall semantic consistency and a local attention branch (Sec. \ref{['sec:local']}) for high-fidelity regional content, boosting high-efficiency and high-resolution video generation.
• Figure 3: Cross-window Attention.
• Figure 4: Comparison results of existing state-of-the-art video generation methods on 1080P video generation. The red boxes highlight zoomed-in regions, where our model produces the clearest high-resolution videos with the most fine-grained details.
• Figure 5: More generated HD videos (1080P & 4K).