Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variance-Reduction Guidance: Sampling Trajectory Optimization for Diffusion Models

Shifeng Xu, Yanzhu Liu, Adams Wai-Kin Kong

TL;DR

Diffusion-model sampling suffers from cumulative prediction error that degrades sample quality. The paper introduces Variance-Reduction Guidance (VRG), a training-free, model-agnostic method that optimizes the sampling trajectory to minimize the weighted sum of per-step prediction-error variances, leveraging a learned mapping fΔ from noise levels to error variance. VRG formalizes the objective as a constrained optimization over the trajectory, solved via projected gradient descent with a regularizer to stay close to the original trajectory; it is applicable to continuous/discrete-time and conditional/unconditional generation. Empirical results across seven baselines and five datasets show robust improvements in FID and related quality metrics, while ablations provide practical guidance on hyperparameters. The approach offers a scalable, plug-and-play enhancement for diffusion-model sampling with real-world impact on speed-accuracy trade-offs in generative systems.

Abstract

Diffusion models have become emerging generative models. Their sampling process involves multiple steps, and in each step the models predict the noise from a noisy sample. When the models make prediction, the output deviates from the ground truth, and we call such a deviation as \textit{prediction error}. The prediction error accumulates over the sampling process and deteriorates generation quality. This paper introduces a novel technique for statistically measuring the prediction error and proposes the Variance-Reduction Guidance (VRG) method to mitigate this error. VRG does not require model fine-tuning or modification. Given a predefined sampling trajectory, it searches for a new trajectory which has the same number of sampling steps but produces higher quality results. VRG is applicable to both conditional and unconditional generation. Experiments on various datasets and baselines demonstrate that VRG can significantly improve the generation quality of diffusion models. Source code is available at https://github.com/shifengxu/VRG.

Variance-Reduction Guidance: Sampling Trajectory Optimization for Diffusion Models

TL;DR

Diffusion-model sampling suffers from cumulative prediction error that degrades sample quality. The paper introduces Variance-Reduction Guidance (VRG), a training-free, model-agnostic method that optimizes the sampling trajectory to minimize the weighted sum of per-step prediction-error variances, leveraging a learned mapping fΔ from noise levels to error variance. VRG formalizes the objective as a constrained optimization over the trajectory, solved via projected gradient descent with a regularizer to stay close to the original trajectory; it is applicable to continuous/discrete-time and conditional/unconditional generation. Empirical results across seven baselines and five datasets show robust improvements in FID and related quality metrics, while ablations provide practical guidance on hyperparameters. The approach offers a scalable, plug-and-play enhancement for diffusion-model sampling with real-world impact on speed-accuracy trade-offs in generative systems.

Abstract

Diffusion models have become emerging generative models. Their sampling process involves multiple steps, and in each step the models predict the noise from a noisy sample. When the models make prediction, the output deviates from the ground truth, and we call such a deviation as \textit{prediction error}. The prediction error accumulates over the sampling process and deteriorates generation quality. This paper introduces a novel technique for statistically measuring the prediction error and proposes the Variance-Reduction Guidance (VRG) method to mitigate this error. VRG does not require model fine-tuning or modification. Given a predefined sampling trajectory, it searches for a new trajectory which has the same number of sampling steps but produces higher quality results. VRG is applicable to both conditional and unconditional generation. Experiments on various datasets and baselines demonstrate that VRG can significantly improve the generation quality of diffusion models. Source code is available at https://github.com/shifengxu/VRG.
Paper Structure (23 sections, 12 equations, 11 figures, 1 table, 1 algorithm)

This paper contains 23 sections, 12 equations, 11 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Diffusion Models
  4. Denoising Diffusion Implicit Model
  5. Sampling Trajectory
  6. Related Work
  7. Sampling with Predefined Trajectory
  8. Sampling with Learning-Based Trajectory
  9. Method
  10. Distribution of Prediction Error
  11. Variance of Cumulative Prediction Error
  12. Mapping between $\bar{\alpha}_t$ and $\Delta_{\epsilon}^{(t)}$
  13. Variance-Reduction Guidance
  14. Experiments
  15. Comparison with DDIM
  16. ...and 8 more sections

Figures (11)

  • Figure 1: Sampling process of DPM-Solver with order=$2$, schedule=$quadratic$ and steps=$10$. (a) Original and optimized trajectories. The former has noise level sequence $\{\bar{\alpha}_k\}_{k=1}^K$ as {0.995, 0.97, 0.88, 0.72, 0.48, 0.23, 0.075, 0.0137, 0.00118, 0.000040}, and the latter has {0.991, 0.95, 0.86, 0.69, 0.45, 0.22, 0.068, 0.0129, 0.00121, 0.000054}. (b) Column 1 is generated images, column 6 is the initial Gaussian noises, and columns 2 - 5 are the intermediate results of the sampling progress.
  • Figure 2: 10-step trajectory $\{\alpha_k\}_{k=1}^{10}$ and learning-portion $\gamma$.
  • Figure 3: Sampling trajectory optimization on different trajectories: $logSNR$, $quadratic$ and $uniform$.
  • Figure 4: FID$\downarrow$ comparison with DDIM on different trajectories.
  • Figure 5: FID$\downarrow$ comparison on different baselines. (a) "ab1" and "ab2" denote $ab\_order$=1 and $ab\_order$=2 respectively.
  • ...and 6 more figures