NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training

TL;DR

NitroFusion tackles the fidelity gap in single-step diffusion by introducing a dynamic adversarial framework that uses a large pool of specialized discriminators attached to a frozen UNet backbone. The method employs a pool refresh mechanism to prevent overfitting and a multi-scale, dual-objective GAN training regime to balance global coherence with fine-grained detail, while enabling bottom-up refinement for 1–4 denoising steps with the same weights. Through extensive qualitative and quantitative evaluation, NitroFusion outperforms existing one-step and competitive multi-step baselines, with clear gains in texture, structure, and prompt alignment. The approach offers practical real-time generation with controllable quality-speed trade-offs and demonstrates strong adaptability to diverse teacher models and styles.

Abstract

We introduce NitroFusion, a fundamentally different approach to single-step diffusion that achieves high-quality generation through a dynamic adversarial framework. While one-step methods offer dramatic speed advantages, they typically suffer from quality degradation compared to their multi-step counterparts. Just as a panel of art critics provides comprehensive feedback by specializing in different aspects like composition, color, and technique, our approach maintains a large pool of specialized discriminator heads that collectively guide the generation process. Each discriminator group develops expertise in specific quality aspects at different noise levels, providing diverse feedback that enables high-fidelity one-step generation. Our framework combines: (i) a dynamic discriminator pool with specialized discriminator groups to improve generation quality, (ii) strategic refresh mechanisms to prevent discriminator overfitting, and (iii) global-local discriminator heads for multi-scale quality assessment, and unconditional/conditional training for balanced generation. Additionally, our framework uniquely supports flexible deployment through bottom-up refinement, allowing users to dynamically choose between 1-4 denoising steps with the same model for direct quality-speed trade-offs. Through comprehensive experiments, we demonstrate that NitroFusion significantly outperforms existing single-step methods across multiple evaluation metrics, particularly excelling in preserving fine details and global consistency.

Figures (13)

• Figure 1: Our one-step diffusion pipeline generates vibrant and photorealistic images with exceptional detail in a single inference step, broadening the potential for text-to-image synthesis in applications like real-time interactive systems.
• Figure 2: Our method distils a multi-step teacher model into an efficient one-step student generator. The Dynamic Adversarial Framework provides dynamic, stable feedback via a large dynamic Discriminator Head Pool, dynamically sampling a subset of heads in each iteration to provide unbiased and stable feedback to judge real or fake, effectively balancing one-step efficiency with high-quality generation.
• Figure 3: Our discriminator employs a frozen UNet backbone with a dynamic pool of discriminator heads. At each iteration, a subset of heads is sampled and trained, with 1% of all heads randomly reinitialized to maintain diverse signals and prevent overfitting.
• Figure 4: Visual comparison of our models (NitroSD-Realism and NitroSD-Vibrant) against multi-step SDXL podell2023sdxl, our teacher models (4-step DMD2 yin2024improved and 8-step Hyper-SDXL ren2024hyper), and selected 1-step state-of-the-art baselines sauer2023adversariallin2024sdxl.
• Figure 5: User preferences study with other baseline models.