PixelGen: Pixel Diffusion Beats Latent Diffusion with Perceptual Loss

TL;DR

PixelGen tackles the challenge of learning high-dimensional pixel manifolds by introducing perceptual supervision into pixel-space diffusion. By combining an $x$-prediction diffusion objective with two complementary perceptual losses—LPIPS for local details and P-DINO for global semantics—the method guides the model toward a perceptual manifold, enabling end-to-end pixel diffusion to surpass latent-diffusion baselines without VAEs. The approach yields strong results on ImageNet-256 (FID $=5.11$ without CFG in 80 epochs) and competitive GenEval scores (0.79), while maintaining a simpler pipeline and reduced training complexity. These findings demonstrate the viability and practical impact of perceptual supervision for efficient, high-fidelity pixel-space diffusion.

Abstract

Pixel diffusion generates images directly in pixel space in an end-to-end manner, avoiding the artifacts and bottlenecks introduced by VAEs in two-stage latent diffusion. However, it is challenging to optimize high-dimensional pixel manifolds that contain many perceptually irrelevant signals, leaving existing pixel diffusion methods lagging behind latent diffusion models. We propose PixelGen, a simple pixel diffusion framework with perceptual supervision. Instead of modeling the full image manifold, PixelGen introduces two complementary perceptual losses to guide diffusion model towards learning a more meaningful perceptual manifold. An LPIPS loss facilitates learning better local patterns, while a DINO-based perceptual loss strengthens global semantics. With perceptual supervision, PixelGen surpasses strong latent diffusion baselines. It achieves an FID of 5.11 on ImageNet-256 without classifier-free guidance using only 80 training epochs, and demonstrates favorable scaling performance on large-scale text-to-image generation with a GenEval score of 0.79. PixelGen requires no VAEs, no latent representations, and no auxiliary stages, providing a simpler yet more powerful generative paradigm. Codes are publicly available at https://github.com/Zehong-Ma/PixelGen.

Figures (8)

• Figure 1: This work shows that pixel diffusion with perceptual loss outperforms latent diffusion. (a) A traditional two-stage latent diffusion denoises in the latent space, which is influenced by the artifacts of the VAE. (b) PixelGen introduces perceptual loss to encourage the diffusion model to focus on the perceptual manifold, enabling the pixel diffusion to learn a meaningful manifold rather than the complex full image manifold. (c) PixelGen outperforms the latent diffusion models using only 80 training epochs on ImageNet without CFG.
• Figure 2: Illustration of different manifolds within the pixel space. The image manifold is a large manifold containing both perceptually significant information and imperceptible signals. The perceptual manifold contains perceptually important signals, providing a better target for pixel space diffusion. P-DINO and LPIPS are the two complementary perceptual supervision utilized in PixelGen.
• Figure 3: Overview of PixelGen. The diffusion model directly predicts the image $x$ instead of velocity $v$ or noise $\epsilon$ to simplify the prediction target. A flow-matching diffusion loss is retained to keep the advantages of flow matching via velocity conversion. Two complementary perceptual losses are introduced to encourage the diffusion model to focus on the perceptual manifold.
• Figure 4: Effectiveness of perceptual supervision in PixelGen. LPIPS and P-DINO losses are progressively added to a baseline pixel diffusion model. The LPIPS loss improves local texture fidelity, while P-DINO further enhances global semantics.
• Figure 5: Qualitative results of text-to-image generation of PixelGen. All images are 512$\times$512 resolution.