Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Critical windows: non-asymptotic theory for feature emergence in diffusion models

Marvin Li, Sitan Chen

TL;DR

This work introduces a non-asymptotic framework to understand critical windows in diffusion-model sampling, where specific image features emerge within narrow time intervals. By modeling data as mixtures of strongly log-concave densities and applying a noising-denoising procedure, the authors derive a master theorem that bounds when a targeted sub-mixture becomes distinguishable from others via total variation, using forward-time separations and score-function analyses. They instantiate these results for concrete cases (e.g., well-conditioned Gaussians and Gaussian mixtures), develop a hierarchical sampling interpretation via a mixture-tree, and demonstrate both synthetic validations and preliminary Stable Diffusion experiments that reveal informative feature-emergence windows. The work also explores practical implications for fairness and privacy, including a new membership-inference attack that leverages critical-window dynamics. Overall, this framework provides a principled lens for interpreting feature emergence in diffusion models and points to potential diagnostic and interpretability tools for real-world systems.

Abstract

We develop theory to understand an intriguing property of diffusion models for image generation that we term critical windows. Empirically, it has been observed that there are narrow time intervals in sampling during which particular features of the final image emerge, e.g. the image class or background color (Ho et al., 2020b; Meng et al., 2022; Choi et al., 2022; Raya & Ambrogioni, 2023; Georgiev et al., 2023; Sclocchi et al., 2024; Biroli et al., 2024). While this is advantageous for interpretability as it implies one can localize properties of the generation to a small segment of the trajectory, it seems at odds with the continuous nature of the diffusion. We propose a formal framework for studying these windows and show that for data coming from a mixture of strongly log-concave densities, these windows can be provably bounded in terms of certain measures of inter- and intra-group separation. We also instantiate these bounds for concrete examples like well-conditioned Gaussian mixtures. Finally, we use our bounds to give a rigorous interpretation of diffusion models as hierarchical samplers that progressively "decide" output features over a discrete sequence of times. We validate our bounds with synthetic experiments. Additionally, preliminary experiments on Stable Diffusion suggest critical windows may serve as a useful tool for diagnosing fairness and privacy violations in real-world diffusion models.

Critical windows: non-asymptotic theory for feature emergence in diffusion models

TL;DR

This work introduces a non-asymptotic framework to understand critical windows in diffusion-model sampling, where specific image features emerge within narrow time intervals. By modeling data as mixtures of strongly log-concave densities and applying a noising-denoising procedure, the authors derive a master theorem that bounds when a targeted sub-mixture becomes distinguishable from others via total variation, using forward-time separations and score-function analyses. They instantiate these results for concrete cases (e.g., well-conditioned Gaussians and Gaussian mixtures), develop a hierarchical sampling interpretation via a mixture-tree, and demonstrate both synthetic validations and preliminary Stable Diffusion experiments that reveal informative feature-emergence windows. The work also explores practical implications for fairness and privacy, including a new membership-inference attack that leverages critical-window dynamics. Overall, this framework provides a principled lens for interpreting feature emergence in diffusion models and points to potential diagnostic and interpretability tools for real-world systems.

Abstract

We develop theory to understand an intriguing property of diffusion models for image generation that we term critical windows. Empirically, it has been observed that there are narrow time intervals in sampling during which particular features of the final image emerge, e.g. the image class or background color (Ho et al., 2020b; Meng et al., 2022; Choi et al., 2022; Raya & Ambrogioni, 2023; Georgiev et al., 2023; Sclocchi et al., 2024; Biroli et al., 2024). While this is advantageous for interpretability as it implies one can localize properties of the generation to a small segment of the trajectory, it seems at odds with the continuous nature of the diffusion. We propose a formal framework for studying these windows and show that for data coming from a mixture of strongly log-concave densities, these windows can be provably bounded in terms of certain measures of inter- and intra-group separation. We also instantiate these bounds for concrete examples like well-conditioned Gaussian mixtures. Finally, we use our bounds to give a rigorous interpretation of diffusion models as hierarchical samplers that progressively "decide" output features over a discrete sequence of times. We validate our bounds with synthetic experiments. Additionally, preliminary experiments on Stable Diffusion suggest critical windows may serve as a useful tool for diagnosing fairness and privacy violations in real-world diffusion models.
Paper Structure (47 sections, 31 theorems, 66 equations, 7 figures, 1 table)

This paper contains 47 sections, 31 theorems, 66 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. General framework
  3. Critical windows for mixture models.
  4. Our contributions
  5. General characterization of critical window.
  6. Concrete estimates for critical times.
  7. Hierarchical sampling interpretation.
  8. Related work
  9. Comparison to georgiev2023journey.
  10. Comparison to raya2023spontaneous.
  11. Concurrent works.
  12. Critical window experiments on real data.
  13. Theory for diffusion models.
  14. Mixtures of Gaussians and score-based methods.
  15. Technical preliminaries
  16. ...and 32 more sections

Key Result

Theorem 1

Suppose $p$ is a mixture of strongly log-concave distributions, and let $S_{\mathrm{init}} \subset S_{\mathrm{target}}$. For any $t\in[T_{\mathrm{lower}},T_{\mathrm{upper}}]$, if one runs the forward process for time $t$ starting from the sub-mixture given by $S_{\mathrm{init}}$, then runs the rever

Figures (7)

  • Figure 1: Cartoon depiction of running forward process for time $t$ (to produce one of the red/yellow/green dots corresponding to large/medium/small $t$) and then running reverse process (trajectories in blue) for time $t$ to sample from some sub-mixture.
  • Figure 2: Example of critical times and proportion in each cluster as a function of noise timesteps. A point belongs to a cluster if its distance to the cluster mean is $\leq 5$. All clusters have identity covariance; cluster $0$ has mean $(-15100)$; cluster $1$ has mean $(-14900)$; cluster $2$ has mean $(14900)$; cluster $3$ has mean $(15100)$. We compute the thresholds $t_1,t_2,t_3,t_4$ with $\epsilon=0.1$ with the formulae from Example \ref{['ex:identity_gaussians']}. By noising for $t \leq t_1$, we only sample from cluster $1$. By noising for $t \in [t_2,t_3]$, we sample from clusters $0,1$. By noising for $t \geq t_4$, we sample from all clusters.
  • Figure 3: Example images of cars generated by SD2.1 that we subsequently noised and denoised to produce Figure \ref{['fig:critical_time_car']}.
  • Figure 4: Percentage of agreement vs. noising amount in the experiment on images of cars generated by SD2.1 (see Section \ref{['sec:sd']} for details). The critical window for each feature is demarcated with double-sided horizontal arrows.
  • Figure 5: Example images generated by SD2.1 from the prompt "Photo portrait of a laboratory technician," that we subsequently noised and denoised for $100$ timesteps to produce Figure \ref{['fig:critical_fair']}.
  • ...and 2 more figures

Theorems & Definitions (57)

  • Theorem 1: Informal, see Theorem \ref{['masters_theorem']}
  • Remark 1
  • Theorem 2: Informal, see Theorem \ref{['corr:wassersteinlower']}
  • Remark 2
  • Theorem 3: Informal, see Theorem \ref{['thm::well_conditioned_gaussian_theorem']}
  • Theorem 4: Informal, see Theorem \ref{['thm:hierarchy_example']}
  • Lemma 4
  • Theorem 5: Section 5.2 of DBLP:conf/iclr/ChenC0LSZ23
  • Remark 3
  • Theorem 6
  • ...and 47 more