Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models

TL;DR

Under mild conditions, it is proved that incorporating diffusion guidance not only boosts classification confidence but also diminishes distribution diversity, leading to a reduction in the differential entropy of the output distribution.

Abstract

Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. Such information is coined as guidance. For example, in text-to-image synthesis, text input is encoded as guidance to generate semantically aligned images. Proper guidance inputs are closely tied to the performance of diffusion models. A common observation is that strong guidance promotes a tight alignment to the task-specific information, while reducing the diversity of the generated samples. In this paper, we provide the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models. Under mild conditions, we prove that incorporating diffusion guidance not only boosts classification confidence but also diminishes distribution diversity, leading to a reduction in the differential entropy of the output distribution. Our analysis covers the widely adopted sampling schemes including DDPM and DDIM, and leverages comparison inequalities for differential equations as well as the Fokker-Planck equation that characterizes the evolution of probability density function, which may be of independent theoretical interest.

Figures (12)

• Figure 1: The effect of guidance on a three-component GMM in $\mathbb{R}^2$. Each component has weight $1 / 3$ and identity covariance, and the component centers are $(\sqrt{3} / 2, 1 / 2)$, $(-\sqrt{3} / 2, 1 / 2)$ and $(0, -1)$. The leftmost panel displays the unguided density. We increase the guidance strength from left to right. This plot imitates Figures 2 of ho2022classifier.
• Figure 2: The effect of guidance on a symmetric GMM: $p_{\ast} = \frac{1}{2} {\sf{N}}(1, 1) + \frac{1}{2} {\sf{N}}(-1, 1)$. (a) In the left panel, we initiate the reverse processes at the origin, and record the classification confidence (measured by the posterior probability of class label) under different levels of guidance. For the DDPM sampler the output sample is random. We generate $10^4$ samples for each guidance strength and plot the averaged classification confidence for both the DDPM and the DDIM samplers, as well as the $97.5\%$ and $2.5\%$ quantiles for the DDPM sampler. (b) In the right panel, we initiate the processes following a standard Gaussian distribution, and plot the differential entropy of the output distributions. For each guidance strength we also generate $10^4$ samples. We adopt the function $\mathtt{scipy.stats.differential\_entropy()}$ from the $\mathtt{scipy}$ module in Python to estimate the differential entropy based on these generated samples.
• Figure 3: Illustration of the negative effect of large guidance. In this plot, we set $\mu = [2, 2]^\top$ in $\mu_{\rm neg}$, and increase the guidance strength $\eta$ from left to right. The upper row uses a relatively large discretization step size ($\delta_k = 0.1$ for all $k \in \{0\} \cup [K - 1]$). Under strong guidance, the center component splits into two clusters at an earlier stage. The bottom row uses a much smaller discretization step size ($\delta_k = 0.04$ for all $k \in \{0\} \cup [K - 1]$); the center component then splits only with a much larger guidance strength.
• Figure 4: Illustration of the behaviors of $h(v_k, k)$ when constrained to the positive real line, under different ranges of guidance strength $\eta$. The left panel corresponds to a small strength $\eta < \eta_0$. In this case, $h(v_k, k)$ is negative and decreasing for all $v_k \geq 0$. In contrast, the right panel corresponds to a large strength $\eta > \eta_0'$, where $h(v_k, k)$ is increasing on $[0, a]$ and decreasing on $[b, \infty)$.
• Figure 5: Illustration of the effect of guidance on a discretized DDIM sampler. For this experiment, we set $p_{\ast} = \frac{1}{3} {\sf{N}}((0,0,0),I_3) + \frac{1}{3} {\sf{N}}((0,\sqrt{3},0), I_3)+ \frac{1}{3} {\sf{N}}((0,-\sqrt{3},0), I_3)$, $T = 10$, and $\delta_k = 0.01$ for all possible $k$. From the plot, we see that the middle component splits with a sufficiently large $\eta$. To summarize, the numerical observations corroborate our theory.

Theorems & Definitions (26)

• Remark 3.2
• Remark 3.2
• Theorem 3.3
• Lemma 3.4: ODE comparison theorem
• Lemma 3.5
• Theorem 3.6
• Theorem 3.7
• Theorem 3.8
• Theorem 3.9
• Theorem 3.10