DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance

TL;DR

DiffuseHigh addresses the challenge of generating high-resolution images with pre-trained diffusion models without training. It introduces a training-free progressive pipeline that upscales images by a noising-denoising loop guided by a low-resolution structure, augmented by Discrete Wavelet Transform (DWT) based guidance and a sharpening step. Experiments on SDXL show that DiffuseHigh surpasses most training-free baselines and matches or exceeds SR methods in high-resolution fidelity and semantic correctness, while offering faster inference. The approach reduces training costs and broadens practical applicability to very high-resolution synthesis and potential video extension.

Abstract

Large-scale generative models, such as text-to-image diffusion models, have garnered widespread attention across diverse domains due to their creative and high-fidelity image generation. Nonetheless, existing large-scale diffusion models are confined to generating images of up to 1K resolution, which is far from meeting the demands of contemporary commercial applications. Directly sampling higher-resolution images often yields results marred by artifacts such as object repetition and distorted shapes. Addressing the aforementioned issues typically necessitates training or fine-tuning models on higher-resolution datasets. However, this poses a formidable challenge due to the difficulty in collecting large-scale high-resolution images and substantial computational resources. While several preceding works have proposed alternatives to bypass the cumbersome training process, they often fail to produce convincing results. In this work, we probe the generative ability of diffusion models at higher resolution beyond their original capability and propose a novel progressive approach that fully utilizes generated low-resolution images to guide the generation of higher-resolution images. Our method obviates the need for additional training or fine-tuning which significantly lowers the burden of computational costs. Extensive experiments and results validate the efficiency and efficacy of our method. Project page: https://yhyun225.github.io/DiffuseHigh/

Figures (13)

• Figure 1: Qualitative examples of the proposed DiffuseHigh pipeline.DiffuseHigh enables the pre-trained text-to-image diffusion models (SDXL in this figure) to generate higher-resolution images than the originally trained resolution, e.g., 4$\times$, 16$\times$, without any training or fine-tuning.
• Figure 2: Progressive High-Resolution DiffuseHigh Pipeline. Overall pipeline of our proposed DiffuseHigh. For simplicity, we did not depict transformation between latent space and pixel space.
• Figure 3: Data sample toward sharp data distribution mode with sharpening. (a) Without sharpening, (b) With sharpening. Red dot represents the data point. We encourage data point to move toward the sharp data distribution mode during denoising process by sharpening the blurry image.
• Figure 4: Qualitative comparison to baselines in 4096 $\times$ 4096 resolution experiment. Please ZOOM-IN the figure in order to see the details of each image.
• Figure 5: Ablating each component of DiffuseHigh. 'DWT' denotes the DWT-based structural guidance and 'Sharp' denotes the sharpening operation. Each sample has 4K resolution, generated from the same 1K image.