Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weak-to-Strong Diffusion with Reflection

Lichen Bai, Masashi Sugiyama, Zeke Xie

TL;DR

Weak-to-Strong Diffusion (W2SD) introduces a reflection-based inference framework to bridge the gap between strong diffusion models and the inaccessible ideal data distribution by exploiting the estimated density-gradient difference between weak and strong models. The core insight is that the reflective update tilde{x}_t = x_t + \sigma^{2t} \Delta t (\nabla_{x_t} \log p_t^{\mathrm{s}}(x_t) - \nabla_{x_t} \log p_t^{\mathrm{w}}(x_t)) steers latent trajectories toward real data regions, with a theoretical basis linking Δ1 to Δ2. Empirically, W2SD yields state-of-the-art or near-SOTA improvements across images and videos, multiple architectures (UNet, DiT, MoE), and diverse settings (weight, guidance, prompts, pipelines), often with modest overhead. The method unifies prior inference enhancements (e.g., Re-Sampling, Z-Sampling, Auto-guidance) under the same weak-to-strong difference principle and demonstrates cumulative gains when combining model-difference types, enabling robust, scalable improvements in practical diffusion pipelines.

Abstract

The goal of diffusion generative models is to align the learned distribution with the real data distribution through gradient score matching. However, inherent limitations in training data quality, modeling strategies, and architectural design lead to inevitable gap between generated outputs and real data. To reduce this gap, we propose Weak-to-Strong Diffusion (W2SD), a novel framework that utilizes the estimated difference between existing weak and strong models (i.e., weak-to-strong difference) to bridge the gap between an ideal model and a strong model. By employing a reflective operation that alternates between denoising and inversion with weak-to-strong difference, we theoretically understand that W2SD steers latent variables along sampling trajectories toward regions of the real data distribution. W2SD is highly flexible and broadly applicable, enabling diverse improvements through the strategic selection of weak-to-strong model pairs (e.g., DreamShaper vs. SD1.5, good experts vs. bad experts in MoE). Extensive experiments demonstrate that W2SD significantly improves human preference, aesthetic quality, and prompt adherence, achieving SOTA performance across various modalities (e.g., image, video), architectures (e.g., UNet-based, DiT-based, MoE), and benchmarks. For example, Juggernaut-XL with W2SD can improve with the HPSv2 winning rate up to 90% over the original results. Moreover, the performance gains achieved by W2SD markedly outweigh its additional computational overhead, while the cumulative improvements from different weak-to-strong difference further solidify its practical utility and deployability.

Weak-to-Strong Diffusion with Reflection

TL;DR

Weak-to-Strong Diffusion (W2SD) introduces a reflection-based inference framework to bridge the gap between strong diffusion models and the inaccessible ideal data distribution by exploiting the estimated density-gradient difference between weak and strong models. The core insight is that the reflective update tilde{x}_t = x_t + \sigma^{2t} \Delta t (\nabla_{x_t} \log p_t^{\mathrm{s}}(x_t) - \nabla_{x_t} \log p_t^{\mathrm{w}}(x_t)) steers latent trajectories toward real data regions, with a theoretical basis linking Δ1 to Δ2. Empirically, W2SD yields state-of-the-art or near-SOTA improvements across images and videos, multiple architectures (UNet, DiT, MoE), and diverse settings (weight, guidance, prompts, pipelines), often with modest overhead. The method unifies prior inference enhancements (e.g., Re-Sampling, Z-Sampling, Auto-guidance) under the same weak-to-strong difference principle and demonstrates cumulative gains when combining model-difference types, enabling robust, scalable improvements in practical diffusion pipelines.

Abstract

The goal of diffusion generative models is to align the learned distribution with the real data distribution through gradient score matching. However, inherent limitations in training data quality, modeling strategies, and architectural design lead to inevitable gap between generated outputs and real data. To reduce this gap, we propose Weak-to-Strong Diffusion (W2SD), a novel framework that utilizes the estimated difference between existing weak and strong models (i.e., weak-to-strong difference) to bridge the gap between an ideal model and a strong model. By employing a reflective operation that alternates between denoising and inversion with weak-to-strong difference, we theoretically understand that W2SD steers latent variables along sampling trajectories toward regions of the real data distribution. W2SD is highly flexible and broadly applicable, enabling diverse improvements through the strategic selection of weak-to-strong model pairs (e.g., DreamShaper vs. SD1.5, good experts vs. bad experts in MoE). Extensive experiments demonstrate that W2SD significantly improves human preference, aesthetic quality, and prompt adherence, achieving SOTA performance across various modalities (e.g., image, video), architectures (e.g., UNet-based, DiT-based, MoE), and benchmarks. For example, Juggernaut-XL with W2SD can improve with the HPSv2 winning rate up to 90% over the original results. Moreover, the performance gains achieved by W2SD markedly outweigh its additional computational overhead, while the cumulative improvements from different weak-to-strong difference further solidify its practical utility and deployability.

Paper Structure

This paper contains 60 sections, 1 theorem, 12 equations, 24 figures, 15 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Method
  4. Weak-to-Strong Diffusion
  5. The Difference of Estimated Density Gradients
  6. W2SD
  7. Visualization and Explanation in Various Settings
  8. 1-D Gaussian Mixture Data
  9. 2-D Gaussian Mixture Data
  10. Real Image Data
  11. Empirical Analysis
  12. Weight Difference
  13. Full Parameter Fine-tuning
  14. LoRA Mechanism
  15. MoE Mechanism
  16. ...and 45 more sections

Key Result

Theorem 1

Suppose $x_{t}$ is the latent variable at time $t$, let $p_{t}^{\mathrm{s}}$ and $p_{t}^{\mathrm{w}}$ denote the probability density estimates derived from $\mathcal{M}^{\mathrm{s}}$ and $\mathcal{M}^{\mathrm{w}}$. The reflective operator $\mathcal{M}^{\mathrm{w}}_{\mathrm{inv}}(\mathcal{M}^{\mathrm where $\Delta_{1} (t) = \nabla_{x_{t}} \log{p_{t}^{\mathrm{s}}(x_{t})} - \nabla_{x_{t}} \log{p_{t}^

Figures (24)

  • Figure 1: W2SD leverages the gap between weak and strong models to bridge the gap between strong and ideal models.
  • Figure 2: The qualitative results of W2SD demonstrate the effectiveness of our method in various aspects, such as text rendering, position, color, counting, and object co-occurrence. We present more cases in Appendix \ref{['sec:exp_res_quali']}.
  • Figure 3: Visualizing the effectiveness of W2SD. When the weak-to-strong difference closely approximates the strong-to-ideal difference (e.g., $\Delta_{2}(t)-\Delta_{1}(t)$ is small), the refined latent variable $\Tilde{x}_{t}$ converges to the ideal latent variable $x_{t}^{\mathrm{gt}}$.
  • Figure 4: Denoising trajectories across different settings (1-D Gauss). The weak model (blue) generates only right-peak data due to missing left-peak training samples, while the strong model (red) produces data between both peaks. W2SD balances the distribution by leveraging the reflective operator $\mathcal{M}_{\mathrm{inv}}^{\mathrm{w}}(\mathcal{M}^{\mathrm{s}}(\cdot))$. Additional examples are provided in \ref{['fig:1d-more-case']}.
  • Figure 5: Probability contour plot and denoising trajectories across different settings (2-D Gauss). W2SD balances the learned distribution, bringing it closer to the real data distribution
  • ...and 19 more figures

Theorems & Definitions (2)

  • Theorem 1: Theoretical Understanding of W2SD
  • proof