DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion

TL;DR

The paper tackles the cost barrier of training diffusion transformers for ultra-high-resolution image synthesis by introducing Dynamic Position Extrapolation (DyPE), a training-free method that dynamically adjusts rotary positional embeddings in sync with the diffusion denoising progression. By analyzing the frequency-wise evolution of the diffusion process, the authors show that low frequencies converge early while high frequencies evolve over the sampling steps, and they implement a time-dependent scaling $\kappa(t)$ to allocate spectral content accordingly across Dy-PI, Dy-NTK-aware, and Dy-YaRN. Empirical results on FLUX and FiTv2 demonstrate that DyPE yields consistent, state-of-the-art improvements in ultra-high-resolution generation (up to $16$M+ pixels) without retraining or additional sampling cost, with stronger gains at higher resolutions and across text-to-image and class-conditional tasks. This approach enables scalable, high-fidelity synthesis at resolutions far beyond training data, offering practical impact for high-resolution content creation and paving the way for future time-aware extrapolation in related diffusion and video applications.

Abstract

Diffusion Transformer models can generate images with remarkable fidelity and detail, yet training them at ultra-high resolutions remains extremely costly due to the self-attention mechanism's quadratic scaling with the number of image tokens. In this paper, we introduce Dynamic Position Extrapolation (DyPE), a novel, training-free method that enables pre-trained diffusion transformers to synthesize images at resolutions far beyond their training data, with no additional sampling cost. DyPE takes advantage of the spectral progression inherent to the diffusion process, where low-frequency structures converge early, while high-frequencies take more steps to resolve. Specifically, DyPE dynamically adjusts the model's positional encoding at each diffusion step, matching their frequency spectrum with the current stage of the generative process. This approach allows us to generate images at resolutions that exceed the training resolution dramatically, e.g., 16 million pixels using FLUX. On multiple benchmarks, DyPE consistently improves performance and achieves state-of-the-art fidelity in ultra-high-resolution image generation, with gains becoming even more pronounced at higher resolutions. Project page is available at https://noamissachar.github.io/DyPE/.

Figures (13)

• Figure 1: DyPE enables pre-trained diffusion transformers to generate ultra-high-resolution images (16M+ pixels) without retraining and without inference overhead, solely by coordinating the positional encoding with the diffusion's progression. We compare the baseline FLUX, YaRN, and DyPE, specifically the Dy-YaRN variant, both applied on top of FLUX, at $4096 \times 4096$ resolution.
• Figure 2: Spectral Evolution of Samples in the Diffusion Process.(a) shows the average PSD of images produced by a diffusion model, as a function of time $t$. The flat spectrum at $t=1$ corresponds to the initial Gaussian samples, and the characteristic natural images power-law appears as the process ends ($t=0$). The combinations of these spectra, corresponding to the mixture distributions $q_t$, are seen at the intermediate steps (gray plots). The progression map$\gamma(f,t)$ from Eq. \ref{['eq:progression_map']} is shown in (b), and measures in relative terms how each Fourier component evolves from pure noise ($t=1$) to its clean image value ($t=0$). As seen in the top rows of this map ($t\approx 1$), the high-frequency modes evolve gradually and nearly linearly across the entire reverse process. By contrast, the low-frequency modes converge much faster and cease to change early on, as indicated by the map's saturation (yellow) in the lower rows ($t\approx 0$).
• Figure 3: Zoom-in comparison at $4096^2$ of Dy-YaRN vs. YaRN. Three magnified regions highlight fine-detail differences. Additional example can be found in Fig. \ref{['fig:zoom_app']} in the Appendix.
• Figure 4: Qualitative results at $4096^2$ resolution using two representative prompts from Aesthetic-4K. We compare NTK-aware, Dy-NTK-aware, YaRN, and Dy-YaRN.
• Figure 5: Resolution scaling analysis.